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PREFACE . * 



This is a study of several scientific ^communities which used a 

computerized, conferencing system for a. period of about two ye^s t<$ 

enhance their communications and carry out cooperative tasks . Though 

it focusses on orie particular ^rstem, it was / designed to yield some 

data that make possible" direct "comparisons rfith the results of 

studies of other computer-mediated communication systems. Included 

are an examination a of the determinants of acceptance of this new i'orm 

* * * 

of communication, user reactions and preferences related to .specific 

. i ■ 

.system features and design choices and how , 'these change with 

experience; and chafiges in communication patterns, *ork patterns, and 
productivity-related measures as a result of using the system, 

i * m , T * * 

■ I m 

The case study should appeal_ to thpse interested In £he applications 
and social Impacts'of coitfputer-mediated communications systems and 
their design fc and evaluation; and' also to those interested in the role 
of communication in scientific research specialties ir^ general, and 
the relationship ijetweeA technological innovation and social change 
in general. ' ■ 

Wi^|iam Why (1980, : 5), defines a social invention as: 4 

...a new and apparently promising Strategy designed to 

solve soijie persistent and serious human problems* It' rtiay 

take the form of a nrew organizational structure or a new 

set of interorganizational relations. It may involve a new 

set of procedures for, shaping human interactions and 

^activities and the relations of kumans to the natural and 
human. environment . * 



Computer-based communication systems, if they live up* to the hopes of 



♦their des*igner?s/are a social inventions in a>\l of these senses* This 
study irs an attempt to describe the •nature anfe impacts of one such 
'case /history in social invention, the Electronic Information* Exchange 
Systern, (S2ESJ, which had as* X$$w "objective the enhancement of 



«^mmunlcati.on 
es\ 



and productivity ♦ Within 



sclent if Lc Research 



Mbyte's prescription for how, a sociologist * \$ to; study ^such an 

Invention was written after this case study wras completed. However, 

' * f * ' % 

it serves to describe the basic approach and objectives tfell. . The 

'Sociologist n^eds^ to .observe and interview; the participants***' 

Then the -sociologist needs to evaluate the effectiveness of 
toe Invention. This m^y involve gathering concrete and a * 
Material Indices of change that can be attributed* to 'the* , 
'invention as: -well as an assessment of the attitudes * and 
- perceptions of 'members of the organization, affected by the 

* Invention* ^ * *\ 

«7 ' , 

• J3Ut that not all. Th£ sociologist can make his most 
^distinctive contribution <In discovering the theoretical 
r principles unaerlying the success or failure or an - 
invention..* *and the characteristics of the social and 
material en^Irqjjraent into rfhiah it must be fitted in-order ' 
to solve human problems* (Ibid) • - % 

"v ..• •-' ^ - 

It./is. the *dif f erenbes In reported Impacts and attitudes among 

y * * 

individuals and m groups using the EljiS ^ystejn that can^reveal the most 

about the" conditions under which computerized conferencing can- 4 solve 

communications problems in the sciences and in other areas' of human 

effort , where, geogcaphically dispersed persons can benefit from 
*' . * * 

collaboration .and the excnahge of information. Thus,*- the analytical 

m ^ ■ 

focus of this report is to describe the important aspects of the 

• » ■ * 

experiences of those who participated in the- EIES operational trials, 

'and to locate factors associated' with significant differences in 



•13 



^success of- outcome* The study will fe^a success only If It serves as 
^the basis for better decisions ( in ( the future aboi*t the ^MLgn $nd 

:aflTor 



social implementation of computer systems for human coiftmunicaflTon. 
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' s CHAPTER 1* 

* * - INTRODUCTION 



1 



This is a study of a new mode of Communication and of its acceptance 
by and . impa£^£ upon a particular user population: groups 'of 
scientists working within the same research specialty. t Computerized 
conferencing systems use a computer to store communications among 
groups jof humans. They can thus exchange Ideas and information on a 
regular, basis without having to be infthe same place or communicating 
at same time. * * * * 



.Among the fundamental characteristics of computerized ^conf erencing as 
a mode of communication are that: 



— One communicates through a computer terminal by typing ancL reading. 

■ 3oth cognitive and 3 ocial -emotional exchanges tend to be different 
tnan face-to-face communication (See Hiltz, Johnson, Ar*onovitch, and 

■ Turoff> -19«0). 

■ * 

— Uonuhunicatlon is "asynchronous", sending and receiving may occur 
seconds apart/ or days or even years apart. 

— The computer's memory can be used to *s tore and find communications 
* and information; one can therefore retrieve stored material by 
attributes such as topic. One can also' filter one's communications, 
deciding whether,, when, and how., thoroughly to choose to read items 



from the mass of material to wh£feh one has access. 



♦ * 



--rhe ' computer can^ be programmed to provide a variety of 
communication structures and -services, such as tabulation a'nd^ctisplay 
of vot£s, or analysis andV dis P la y of information 'according to a 
format specified' ."by a particular individual or: group (See Hiltz and 
Turoff, 1978*, Chapter 9)- . . 



The first computerized conferencing system was designed and 

implemented in 1^70 (See Turoff, 1972). There are now hundreds of 

* * * ■ 

systems which 'use tiie computer* to store and mediate human 

communication, most| of which arej very simple message systems or 

Community Bulletin Board Systems.^ The Electronic Information 

Exchange System 1 ISISS? is a computerized. -conf erencing system which 

was originally. designed specifically to enhance communication within 

"geographically dispersed "small research communities" of scientists, 

"conceived as groups of 1Q* to 50 ihdividua]^ sharing an interest In,. a 

scientific or 'technological problem area" .(NS? 7b-^5 : 3 ; • EISS 

provides a message system • which * enables members to send private 

A • * 

communications tro individuals o*r groups on the system, "conferences" * 



which build up a permanent transcript* on a topic of discussion, and 

■ 

notebooks where scientists may use text processing features to work 

>* • 

on jointly authored report^. It also provides the capability to 
create special structures to handle unique kinds of information or 
knowledge oases, or to change the interface, or conduct a controlled 
'experiment. * For example 1 , one of the scientific groups in this study 
had capabilities designed for 'them to produce an "electronic 
Journal." , Another gr^oup had software designed to facilitate 
inquiry-response r exchanges -that followed £ selective tree-like 
structure rather than the linear transcript structure of the regular 
conference s/Steijj (See'' Johns on-Le/iz, 1981). p 

•o •' ' ' . - 
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The Division of Science Information (*now the Division of Science arid 
Iitformatiogi Technology; of the National S'cience ?oundation issued a 

" i ' ■ 

prc^ram announcement In 1976* inviting proposals for "operational. 

• trials" cjf EIES* , Pour groups were chosen to participate beginning in 

• * * • * 

* ■ * * * 

. late 1977; three final groups were chosen in 1978. In addition, 

■ m * , 

several "other groups 1 made use of the EIES system jti&x DIST 
t * 

permission, but without- DiiJT support • . 

^The off„iciVi objectives of the Operational Trials program were: 

r 

/ — to 'test the hypothesis that EIES can enhance the effectiveness -of 

- 

Individuals belonging to 'such a community. 

H 

—To* accumulate practical experience with EIES by 'the members of m such 
a* community. 

— To gain deeper iifeight into the relationship between cijf^muni cation 

processes and the progress of science and technology 'INSF-76-45 . • 3) 

t • / 

The Division ot Mathematical and Computer Research funded a study by 
the author of this report to conduct* an cross-group assessment of the 
impact^of t?he use of EIES,^ with the*f ollowing objectives; 

j ' ■ ' ■ ■ 

—Feedback to the designers on user reaction to specific features of 

v. t 

the system ' , 

7 

— Isolation of the factors accounting f$r low vs. high levels of use 



Identification -of the individual and group impacts ,of the system on 
those; ^ho do make substantial use of it t * * * 

It ftas hoped tp make the study* comparative axross systems. ^ One- other 
scientific user cojnmunity on MACC-telemail, theoretical computer* 
scientists,, did agree to participate* However, response rates from 
that "gr.oup wereYairly low. A brief description of that system* ^nd 
tne results for the Theory, yet group will be reported in a separate 
chapter. In addition, indi^ScT comparisons to the PLANET and NLS 
systems are made possible bousing some *of the same questions for 
users tnat had been employed in earler studies of these systems. 
• ■ 

In this introduction, we will look at previous findings about 
scientific communication ^hich formed the basis for £he variables 
examined in this study". We will** also include a' more detailed 
overview of the EIES system which served as a oontext for most of the 
data collected, a summary of the nature of the scier^tif ic user 
groups, and a description or the evaluation research methods used. 
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-THEOKbTlCAL WAMEWORK AND INITIAL HYPOTHESES % . 

V • . . • . 

/ 

The model of analysis, begins with several "input" or independent 

variables: characteristics of the individual user; of the scientific 

user . groups and the tasks they undertook on the system; and 

characteristics of the system Itself. This framework was Initially 

developed by Jacques Vallee and his coll£agues~at the Institute for 

tHe Future i vallee et al. , 1974:22} In their evaluative rffcrk on the 

PLANET computerized conferencing system. It Is'des£ribed In detail 

in the final report or^ another project rfhlch was undertaken; 

concurrently with this/ one IHiltz and Xerr, 198l>« Among the 

characteristics of Individuals which were measur£ci are skills (such 
■ * *** 

as typing and previous computer experience), initial attitudes toward 
■the system'they were invited to use, patterns or communication and 
exchange with .other scientists in. the specialty; and access to 
compter terminals. Among the Important characteristics of the "group 
are Its. size,, cohesiveness , leadership, at*d the nature 'of the task it 

' is tcxing to accomplish through using the computerized conferencing 

v ' / 

sjrstem. . . / 

The system Itself has, characteristics such as ease of learning, the 

nature and quality of documentation and training help offered to 

« * * 

users, the "friendliness" , of its Interfaeg, and ■ the nature and 
variety of capabilities which^it offers-. (See Hiltz and K^rr, 1981, 

^ptbr' 2, for ~a complete rptfiew of the variety of system 

characteristics and their relative importance for user acceptance) . 
As a < result of the interplay of k individuals groiip and 4 system 
attributes, 'individuals make choices about whether or not to use the 



system at all. Some become dropouts, some become "addicts' 1 who spend 

• * . I * 

Several hours jja day woriclng and communicating on line. Through 

* N • * • y 

systematic feedback, ttye system itself undergoes change. The 
individual^ f aad user groups aVfo change, as a function of how much 
they use the 3ystem« This study collected data over time /which could 
txe us*ed to track this complex process. Subsequent chapters will 
include discussion and review of the literature on individual, group 
\n& s/stesi attributes wh^ch seemed to have important influences on 
the nature and degree of r impact of EXES. 



The sections which follow summarize tne research which was ased in 
conceptualizing the -potential effects of ,the use of EIES upon 
scientific research communities. They are <i?a.wn from the original 
proposal to the National Science Foundation,' and represent the 
background Soxf and justification of scientific research, communities 
as tne initial population for a study of the potential impacts of 
computerized conferencing systems*. * 



unaracteristics of Scientific Res 

c 



r 



arch Communities 



A. SCIENTIFIC COMMUNICATION 



cientific specialties consist of a set qf scientists who engage in 



research along - similar lines and 
intensively with one another (Hagstrom, 
(1975: 1) has pointed out, "disciplines 
science, while smaller intellectual unit;, 
disciplines) comprise the research domain 



who communicate often and 
970: 91-92*. As Chubin 
form the teaching domain of 
(nestled within and between 



A 
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' Such specialties have sometimes i>een called, "invisible colleges" of 
scientists (Price, 1963; Crane, 1972)' and have* b£en seen as the social 
location of technical, Cognitive, 7 and , ethical norms (Mulkay, 1972; 
Mitroff, 1974a) and as Internally stratified on the basis of 
productivity (Cole and Cole, 1973)- f ^ m \ 



Geographically dispersed networks pf scierrtists working In the same 
specialty area can be v.iewed as the prototypical m "production 
organization" of science, in which the "product" Is ^scieritifT£ 
knowledge, and the social organization depends almost entirely upon- 
the communication system. Not only do the formal and informal 
communi cation system ssrve to direct and redirect efforts \ to 
^important" areas ana the most • rruitrul metnodological tools y *ut 
tney also reint'orcg shared norms and theories and alloc&tes rewards 

* w 

In the form of recognition. . « 

Cole and. Cole 1,1973: 1b J describe'the importance of communication in 

^science as follows:* 

"Scientific advance is dependent on tire efficient 
communication of ideas. The communications - 
/ , system then 13 the nervous system of sciencS; the 

system that deceives and transmits stimuli t*o 
its various parts. 1 ' ■ 

\ ■ 

* The actual processes through which this crucial formal and informal 

I cpmfiiuni cation t^kes place have not changed in decades except that in 

many disciplines, an exponential growth has slowed down the, process 

and lengthened the time between the completion of a research project 

_/tnd its publication in a Journal. Summarizing the results of a 

series ' of studies of scientific communication in the discipline of 

psychology, which is Similar to patterns in mafty other disciplines, 



Qarvey and urirnth t^S)71: 354, 355) conclude that the scientist 
relies heavily on informal networks or discussion, small meetings, 
"and exchange of drafts and ^preprints * to keep abreast of current 
activities and of the cuprent^views of the community on the value and 
relevance of specific research problems. The Journal article, by the 
time it, is published, lags so fjar behind the research frontier that 
its functions are mainly to. inform scientists in other specialties, 
and to allocate recognition for scien&ifi^ achievement. 

* * 
Increasingly, there have 'been calls for improving scientific 

comiaunication and information dissemination. Many of** these have 

focused on the information Storage, processing, and networking 

capabilities of the. computer to provide assistance. 

Some of the suggested innovations deal with the -formal communication 

channels, the professional meeting and the journal. There are 

estimates that there may be 100,000 journals publi-shed in 19tfQ; 

something must be done to decrease the costs and Increase the 

efficiency of dissemination* of "published 1 * results. Selective 

dissemination of articles only m to consumers^ ho peruse computerized 

ibstracfts and order a copy *■ of the full paper has been one , answer; 

another has been more \efficient, computer-assis'ted publishing 

procedures (See Rhodes and Bamf,ord, 1976). * 

^ ' x , • : 

Another approach has been -to make ■ Scientific* information, 

particularly In the form o!* .data bases and bibliographic files, 

directly available to' researchers through *an on-line, interactive 

^com'puter system. One example of this is the NIH-EPA " Chemical 

literature, on a central computer which can 1 ije accessed from 

, 4 « 



. * telephone-coupled computer "terminals ahywhere in the world. " The user 
searches and retrieves .Ihf ozonation and performs data analyses on 
these files through conversationally-designed computer programs. 
*(Se^ Heller, et'al«, 19.77). Scores of abstracting services have also 
been computerized and programmed to allow a person to Interact with a 
compter to search these flies using combinations of key. words. 

However, the informal pre-publLcat Ion .'communication within 

I 

scientific* specialties is also crucial* to increasing scientific 
productivity- Recognizing this, the Division of Science Information 
of the National Science foundation financed the building and field 
testing of EIES as a computer-based communication system designed 
* - specifically to ' meet the ne.eds* of networks of geographically 
§ dispersed scientist*. > t , 

- • . , 

3. POTENTIAL IMPACTS 0? COMPUTERISED CONFERENCING Oft COMMUNICATION 

r • 

•AND PRODUCTIVITY ■ v 

xt was hypothesized, that there would be 3ome very marked affects of 

the use of computerized conferencing upon the scientific specialties 

which utilize it. Consider that existing communications structures* 

are either * yery slow (printe<ff journals) , very fltfull and expensive 

-(yearly conferences , or special? meetings) , or very excl usl /e (personal 

letter, personal visit or telephone call J., Computerized conferencing. 

could, enable the members of a user group to keep In constant 

communication with., one Another and to exchange ideas and f lndings- r on i 

* * * 

a daily-to-weekly bas'is, sending "and receiving such materials at 
* * 

the>r own convenience. It could Increase the amount w and timeliness 

of . the raw materials (information and Ideas) ^sed in the scientific 

1 V 
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process and thus increase the productivity of scientists. 



4 



IMPACT UN UUVbLOfMbNT UND RESOLUTION OP SCIENTIFIC CONTROVERSIES 



We were especially Interested in studying the Impact of computerized 
conferencing upon research specialties in which there are s.ome^oasic 

* . \ - m 

theoretical 1 conflicts or controversies, with the competing theories 
each paving tJ>eir adherents. Often this will ociur when #arge 
amount? of new data or new types of data are becoming available. 

. • " / 

. i 
Studies by distinguished analysts or science such a^ Kuhn, Mertorf, 

and ^eyerabend hav*. established .that controversies are a perpetually 



recurring, If not permanent, feature of science. Such studies also 
establish that controversies are . a vital feature of science in the 
sense that science is fundamentally dependent upon them .for the 
int^rojection' of fresh points jof view and the challenging of old 
established beliefs. In other words, it is expected that in the 
napuril course oi development *of science that scientists of different 
"schools" of "thought, theoretical persuasions, points of view, and 
disciplines will develop different hypotheses with regard to the 
$anie phenpmena.. It is also ,to be expected* that some of tftese 
hypotheses will clash .sharply, since they are frequently based on 
different ideologies (see Bobbins and Johnson, 1976)* For this 

* r 

reasor^ scientific groups are espeqially likely to be affected by the 
us§ of i computerized 'conferencing if they are about to experience 
sharp clashes of opinion /within the particular group or ' the 
discipline as a whole with regard . to an Important problem area of 
conctfrn. • ' . 
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One can imagine the emergence of a new paradigm as a kind of Hegelian 
process. A new theory or Method may arise to challenge the existing 
dominarrt approaches. There may be a periQd or increased controversy 
as the two sides argue. Then the controversy may be resolved by some 
sort of synthesis of opposing points^ of view. ; v 



v 
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D« IMPACT UVON THIS OVERALL STAGE OP .DEVELOPMENT OP , A SCIENTIFIC 

■ 0 ' • ' ' ' ' ' 
SPECIALTY - N ' » * 

Related to the development of scientific cpn'troversies , it * was 

hypothesised • that' there will be a differential impact of the 

intensified communications made possible -by computer conferencing 

* i m 

upon scientific fesearsh communities', depending upon the stage of ■ 



development - of, .the specialty when computerized conferencing ■ is 
introduced, Thomas xuhn has formulated widely used ideas atfotit the 

. 

nature* of the differences among the sciences', which begin with the 

premise tha£ a fully developed specialty area has a fully develgp4cS 

and fully shared ^ "paradigm." * In the study this is tapped by 
• » • 

questions referring to whether, or 'not 'there is "intellectual 
mainstream V M ■ ' 



\ 

Following the analogy or the Hegelian dialectic one can hypothesize 

that the first step in the development' of a shared paradigm or a new 

paradigm is - the sharpening pf methodological • or theoretical 

controversies. The second stage might be their resolution or 

"synthesis" into a new "paradigm" # or "mainstream*" One would n9t 

''expect such a full cycle to necessarily be completed in eighteen to 

« 

twenty-four months • p Thus, this study looks Jt'or both parts of the 
hypothesized process: the clarification of controversies and their 
resolution. ■ ■ *. 

Among the other specific questions related to growth and change in a 
specialty,, which can be explored is whether a C.C« system can increase 

ERLC • 12 



'the* motivation or" probability Jjf^ja siientist's contributing ideas for 
a piece of work whicft another Scientist has in the formation stage. 

j / , • • 

* A standard view or* motivation in _^scl%iftre has the scientists. 

exchanging "gifts" of" published results' for the reward' of) 

/ / 
recognition.* "One of th,e norms of science i5^tJia**once something has 

been -published, thosfe who use it are supposed to acknowledge its 

source. Informally "helpful" information, typically # exchanged at 

conferences ot? in conversation, - however, frequently is not 

acknowledge^ - pos^ibly"because the scientist who received the insight 



r 



or advice £o&$&z its soyrce. E1ES prq.vlfces the dafe and tinTe and a 

.^written recora of all Suggestions °r advice; thus it' might become 

easier and much more expected 'that the recipient of such material 

will acknowledge ' its influence when ^th% results finally are 

published. T&lk greater probability of this formal recognition for 

such cjontributiohs to the research of others would, in turn, increase 
# #■ 

the motivation to engage in the activity. 



. On the other hand, it may be that scientists will be very reluctant 
. to 1 maice detailed suggestions about the research projects discussed by. 
athers, because or^the^ lacK of apparent reward* TdB doing so; or to 
enter their own research plans and problems, for f,ear that they may 
* be "'stolen" and 'published by. someone else. 

J — " gr — — " — ■ 

. , ♦See-^'or instance, Hagstrom, 4 190t> and Stoker, 19bb. ANiQholas Mullins 
has pointed «DJifC that perhaps if the Tnetaphor is to be applied at all, 
It 'is more like a potlatch or a frenzTed feeding of sharks than a 
polite d&change (private communication). 



r 
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THE SOCtOMETRIC STRUCTURE 6P SPECIALTY AREAS 



i 



Another area of inquiry is the impact of conferencing systems upon 
the size of the specialty. In regard to- the size of the' group of 



actively communicating an^working scientists wi£hin a specialty, for 

example, will computerized conferencing condense the research 

specialty, so to speak, into a smaller pore group, with those not in 

the system more completely ci^t off? Or, will the increased ease of 

communication witftir^ this pmze facilitate^ expansion through 

circulation of some ^of the printed output, invitations to "observers fr \ 

or VT visitors" to occasipnally take part, the freeing of time to do 

* 

more letter-writing and manuscript circulation to more people, and/or 
the facilitation and Inspiring of specialized f aceJ-to-f ace 
Conferences to which a general invitation is extendejp ^ 
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Scientific research ^ cpmmunities are not only networks' of 

^.ommunication, but are also stratified social sy3yems which allocate 

prestige and opportunities. For example, as Price and Beaver^ 

.(^269:101-117) describe their concept of invisible colleges: * 

The basic phenomenon seems to be that in each of * 4 

* the more actively pursued and highly competitive » f - \ 

of the science^ there is an * "in-groujp. M The 
1 people in 3uch a grtfup. claim to be reasonably irj c ' ' , * / 

touch with everyone else who is contributing 
materially .toresearcfr in this subject not merely 
on a national scale, but usually including all % „ * 
other countries ■ in which that specialty is" 
strong. Tne Aody of people \rneet in select 
conferences (usually held «in "rather pleasant 

places), they, commute between one center and A . 
another,, and they circulate preprints 1 to each 
* " other and they collaborate in research. Since 
, , they constitute a power group of everyone who is 
• * really somebody * in a field, they might at the 

1* ■ ' 

■ 51 * 



/ 
\ 



. * 




£pcal and national level, actually control the 
ml^nl^rariqj^f * research furjds and laboratory 
races. 3#ey may also control personal prestige 
>and the 1 rate or new scientific ideas, and* 
^-intentionally or unintentionally they -may decide 
/the general strategy of attack in an area. 



Two i ntere sting inadequacies^ of the ^invisible college" structure are 
immediately obvious. First, for those who 'are "in," the existing 
cofamunicat ions . network is so time-consuming, sporadic and slow, that 
only a few of the many, questions, answers and comments that might 
.fruitfully be exchanged actually are. . Secondly/" what about those 
potentially productive scientists who are "out"? An analysis of, 
productivity patterns of chemists tReskin, 1977:441)) suggest that 

"collegiate recognition is particularly important for ^chemists in 

A " * 

contexts that that do not stress^s^holarly publication." 

A computerized conferencing systei\ might make the exchange within 
"in" groups more effective. it ai-so could allow the rapid formation 
of communities that ~3o . not now exist. * A grfcup of younger unknown 
researchers could form their own p^r group > independent of the # 
"established" in-group. Moreover it could/ allow ^new modes of 
interaction between "elites" and, "newcomers" (see Mulkay, 1976, for 
one view of current relationships). 

Thus ari issue of interest is the question of which types of 
scientists can be most aided by such a system, those who are already 
part of a highly productive elite within a specialty, or those who 
are currently cut off from opportunities for extensive communications 
and cooperation with ottters in the field. At present, the. academic 
community^ is very much a stratified orte, with those scientists 

/ ' ... 
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located at the * top universities having a much greater opportunity to 
be productive and gain recognition, because more time, money ana 
equipment is available ror research, and^b^ause of the greater 
likihood tljat their academic affiliation will automatically include 



i them in an e^Jting communication network.. This is an example of 
what Merton' (1968) ca^s the "Matthew Effect" in science, quoting 
from* the 3ook of Matthew: "For untbv everyone that hath shall be 
given, .arid he ^hall nave abundance; but.f/rqm him that hath not shall 

.be taicen away that vtticft he ftath*" v 
* * 

AiiisoA ana ( Stewart ^197^J rf have used cross sectional survey aata to 
provide eviaence .that at least for f chemists7~* - ph>ysicists , f and 

mathematicians, getting <?f f\ on "the wrorrg foot"* can severely lessen 

# V \ 

the opportunity t to jever have the 'contacts and resources to. be 
"productiVe^fin terms of research. They ^summarize their findings as 
follow^: K 

> The highly skewed distributions of productivity 
among scientists can be partly explained by a 
' " v proc'ess of accumulative advantage, Because of 
4 * feedback through recognition ?tnd resources, 

highly productive scientists maintain or increase 
'their productivity, while scientists wfio produce 
yeYy; .little produce .even less later on, A major 
/implication of accumlative advantage is tfiat the 
,« distribution of ^productivity becomes increasingly * * 
* , unequal^s a cohort of scientist/8 ages. (p. $yb) 

* ' '* 

' 1 • 

It is poSsioie *. that a computerized conferencing system can provide 

< 

equality of opportunity amdng research members "located at "small*' 
colleges Mho are "unknowns , 11 with those at maj.or' institutions, 1 It is 
also quite possible that the researchers at small 'inatitup ions would 
benef.it more in -terms j>f productivity by the increased stimulation 
due to improved peg^ group communications. » 

r!c . « . 30 ' . \ • 



To examine these issues, the study includes som§ pre-use measures of 



th(.; 




productivity and of how well known the .scientists are (subjective 
assessments)* Thestf will be looked # at in terms \>t their relationship 
to amount 6f use of the system and to Subjective reports of the 
impact of*systera use on productivity and on the resolution of 
theoretiqal or methodological controversies. " 

THE .STRUCTURE 0? EIES 

EIES provided four general purpose structures for all its 

* 

. * MESSAGES: The delivery of messages to individuals^a^d/or defined 
groups. This facility includes confirmations of. delivery, a "central 
message file* editing, retrieval, searching and resending, ars well as 
historical analysis of message traffic by individuals. > 

" CONFERENCES: Linear time sequential transcripts" of group 
discussions on . a particular topic with status information on 
^adership. This facility* includes voting, tejtt searches,, automatic 
delivery of 'new material . to individual conferees and other 
communication sbj^pa^t functions. Descriptions of opep conferences 
are llst ^^ in a pu6lAc conference) and an individual may Join any 



number of conferehces. ^ 



NOTEBOOKS; A text composition and word processing space that may 
be private to an individual or Jointly shared among a group of u§ep*-r 
Provides features for organizipg and distributing documents as well 

* / \ 17 
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• * i 

as automatic notification to users of edits and .mod if icationsj 

v 

r 

DIRECTORY: A njemfe^ershjlp, directory containing both individuals and 

defined groups with self entered interest descriptions and numerous 

search options. A defined group may be .treated as a single 

-/ 

individual for purposes such as sending a message. 



Messages ane either ^private or group messages, and commences and 

■ 

notebooks may either be private, group or public.^/ Private 

conferences arid notebooKs are controlled by an individual user rfho 

'determines the participants. 3roup conferences and notebooks are 

controlled' by defined groups on £I£S, 'while publi-c conferences or 

notebooks are available to anyone on <the system for reading. Public 

notebooks have a defined ^et of" authors (.restricted writing), but 

* 

anyone can read in tnem. ' „ 

/ 

• * 

All the tdxt items in the above subsystems are compatible and reacjily 

* •* J 

* 

transferable, i.e., a message may be transferred^ into a conference 

comment or notebook; *-page. f . All of_the subsystems exist within the 

* * j 

context of a single user ,in££rf aee that provides four different modes 

L^tebface rpo 



of user* interaction. The user 'iijtebf ace rpodes are 



MENU ELECTION : the user selects an option from a list included on 
the one-page guide to the main, set of SIKS menus. 

. . COMMAND DRIVEN: all the menu selections are available as commands. 
In add^ion, approximately 200 advanced features not available, in the 
menus can be 'Utilized. * • ' 



^ ANSWER AHEAD AND^. COMMAND STREAMS: The user can anticipate 
questions arid answer ahead or ^rigger a sequence of operations* The 
i EISS interface, is fully predictable to. the user and all commands are 
usable at apy point Iji the interaction. 

SElil^UfcKifiliU COMMANDS: tjie Individual user or a group coordinator 

can derine commands unique to the. Individual or group* There are 
L 

facilities for defining commands that /fill accept input* control at 

* 

\the tifte they are executed. , » 1 

» * * m 

V * 

■ 

In addition to the above, ZIZS f has, a general purpose ^language 
(INTERACT) that can Interpret any input stream from a user or from 
EIES as an executable program.* INTERACT programs are ?tored in %itS 
text items. mis caoaaiiity allows selective tailoring of the 
interface and communication features of „ EIES. J>s|[ individuals or 
groups. * with INTERACT, specialised subsystems are tailored for 
specific applications. Access to a specific EIES program is given by 
readership privileges on the^text Item in rfhich it is stored. 

EIES operates on a 'dedicated minI-computer~an INTERDATA 7/32 *ith 

half a megabyte or core Stnd two^juu megaby% ^isScs. it currently 

■ 

supports up to 32 simultaneous users. SIES n is. Implemented in 

PORTRAY, with modifications tD the compiler .and. to the executive 
system 1 . It is accessed either by a direct telephone call, or through 
the TELENET packet-fswitched network. k TELENET had n<jdes in 
approximately 185 &.S. cities during the period of this study; the 
cost was 33*75/hour to* connect to 1 rom any or these nodes. 



Witnm the Dasic structure of EIES are many specific system features, 



X 



Many of these nave been subjected to u&er ejvdluation, reported later. 
Table 1-1 provides a brief -description of vlarlous system features and 



indicates, which have evolved 1c#fcr • the . 



op 



systep froifi la'te" 1976 to i960. The f 
Constantly evolving, partially as a result 
stuay, greatly complicated the prcrbLem .0 



x 



erational period of the 
act that the system was. 
of feedback from this 

* % 

getting comparable dat& 



*rom users and user groups who joined the syptem at different times. 
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TA'BLE 1-1 



BRIEF EXPLANATION OP EIES FEATURES 



FEATURES IN -THE ORIGINAL DESIGN ■ « 

(An * indicates the feature has undergone extensive additions or 
modifications over the four year operation of the system) 

i Private Messages: Can be sent , to any individual or list or 
j individuals* * confirmation or date and time or delivery is 
given* 

r 

Group^ Messages; <Uel\ivers the' message to all members of a 
pre-defined gtVoup. No confirmations are ptrovided, # but 
sender can request status list showing who has received it. . 

Membership Directory: Self-entered short description and address 
for all groups and members. t Specialized searches are 
( incorporated. * * , 1 ^ 

Private Conferences: Any member may initiate and moderate a 
conference on any topic. m Member has right to involve" 
whatever participants he/she chooses and decides whether or 
not to advertise 4 . / _J • * \ x # 

Group Conferences: Each Group has a permanent general conference 
to which all group members belong. ' ' 

Public Conference's*: Conferences in *hich anyone on the system may 
read or write without having -to be granted access. 

Private Notebooks*; Each member has* a notebook for composing and 
( storing items. The owner of a notebook may give other 
members privileges ~to either read only or write as well. 
Owners may also establish read-only windows to portions .of 
the notebook. New items as well as modifications of 

existing items are reported to all merpbars in a notebook. 

. 

Group Notebooks*: Same features as private notebooks, but 
associated with all members of a group. t 

Public Notebooks*: ^nyone on the sy^tenP may read in a public 

notebook, but' only the designated authors may write in the 

notebook. • 
* c_ 
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Mfenus: ifte standard forja of person-machine Interface taugKt to new" 
users via the written documentation they Initially receive* 

* ; \ \ 

Commands*: System-wide* cormtiands allowing the complete replacement 
6? the use of menus c and adding other .unique capabilities 
outside those available through the menu. 

Explanations: -An on-line searchable rile containing* specific 
*S explanations of all system features. 

Retrieval: The ability to^recall any text Item previously read by 
a unique Identifier. ?or messages . this Is limited to the 
last 30,000 sent on the system (about three months f 
'traffic) "and for conferences or notebooks this Ls based 
upon owners of these spaces deleting Items #hen they are 
outdated. / * 

Searches*. Messages, conference comments and, notebook pages may be 
searched by author, editor, dates, Item Identifier, free 
, - Icey words, full text, associations, among Items In either a 
^nested or Combination process. t - X 

Anonymity & Pen Names*: Any text Item may be signed anonymously or 
with a unique secret pen name. Messages may be sent to pen 
names. 

Synchronous Conferences: The* ability to hold a conference when all 
* members are on line a£ the same tim'e by supplying status 

indications of everyone's position in the conference at any 
time. , v > 

)/ 

Voting*: rhe ability to chVose any * one or two of nine alternative 
* voting scales 'that can be attached to a coherence comment. 

The computer collects^ and displays the vote distribution 
1 for the members of the # conference. 

Direct Text Edits*: A llne-ordented : editor for use in the 
. scratchpad, where Indftviduals ^compose text items for entry 
Into the system, tdits are accomplished Immediately. 

Copy, Get and See: Methods of Indirectly referencing other Itema 
* of text within a given text Item or of transferring te^B^ 
Items among message^,* conferences and notebooks. In the 
case of "See, 1 < t,he ^printout 'of^an Item Is conditional on 
whether the receiver has already seen It. 
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Table 1-1, Cont. 



0 

2V0LVED FEATURES ^ Those added to the EIES system based upon 
• feedbacK rrom usws) ■ * 

• ■ 

User Consultants: volunteers who help others to learn to u'se the 
-system and who also serve as Information brokers on 
activities taking place on EIES. A nupiber of special - 
purpose software features exist to facilitate .the tasks of 
* the user consultant^* 

a 

! 

CHIMO (newsletter) : A weekly summary of events taking place on 
EIES. ' 1 



or Entering a ? or ?? as an answer to any question or choice 

on SIbS results in. a short or long explanation, 
respectively/ ■ . f> 

Vwora: rfiil retriev;^^an explaaation\ of the "word" or system 
feature named^H|om the explanation file*-, - • 

SEN,' V V V or L1MK: Sending one-line messages which are delivered 
the next time' the reci^ent does a carriage return, rfith or 
** without confirmations or <x2ntinuous exchange of one-liners 
with a group. 

Defined Commands: Any user may define a sequence of operations or 

commanas^as an individually tailored command, facilities 

, -^€x^3t for the more sophisticated user to make these 

* * ^conditional. > 

j 

Indirect Stilts: Edit commands stored within the* text providing 
such things as centering, paging, text Justification and 
tabulation. Indirect edits are executed at output, time and 
are based upon the specifications the receiver m has 
indicated for his or her terminal or local interface 
device* 

Storage Areas: # A set of six' temporary scratchpads in which users 
may store fragments* of text undergoing composition. 

< 

Terminal Controls: The ability of a user^to control a margins and 
page size* 

* < 
Switches: Special controls needed to regulate the # output for those 
interfacing through microcomputers and intelligent 
terminals-. 

Reminders: A personalized file of^9ne-line reminders kept by any 
, member which may also be "alarmed" by date and time. 

Interests: A file of key words such as "ham radio" which users may- 
enter and associate with so that messages c£n be sent to 
. all those on the interest list. 
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Submit & Read: t The ability to provide abstracts to others via 
1 messages or conference comments which are active keyholes, 

* upon, demand, to larger documents stored In notebooks. 

Subaccounts: The ability of a: group of users to share a singly 
membership slot wnere^only one of tike group may be active 
at any one time. > * • 

Games: various computer games Incorporating the ability of, players 
to contribute material * to the game or having a 
communication component, (e.g. bridge). 

- ■ 

Graphics: The ability to specify simple diagrams through a size 
independent specification of figures , together with an 
ability to move windows around in a text Item and Insert 
text In windows horizontally or vertically. 

Special Programs: Tailored * routines for specific purposes. For 
example, "Terms" cdllects votes on alternative definitions 
for tasks such as standards setting. "Respond" administers 
surveys with multiple choice questions. V — ^ * 

j 

Special Communication Interfaces; Tailored communication 

> structures such as TOPICS to " deal witn Inquiries and 
Responses within a group and allow members to set profiles 
of. their: interests for self-filtering of the Incoming 
material. 

s m 

INTERACT language: A programing language allowing the imposition. 
* of special communication or data structures on the basic 
EISS facility. ' J ■ s * 
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* DATA SOURCES- AND METHODOLOGY 

* * \ * 

Three sources were originally 1 planned ^rid .have been utilized. In 
addition, threQ data sources have been devised using rthe unique 
caoabilities or ElfcS. All participants in the study were fully 
informed about the purpose of the study and the data that tfould foe 
collected. . Formal "informed consent" agreements were obtained, as 
required by federal guidelines for the protection of hura^n subjects. 

\ 

1. * v Mailed -Questionnaires. These each took -twenty to thirty minutes 
to complete. They were sent out "pre-use," with a first follow-up at 
three to six months and a second follow-up at eighteen months. 
latter is referred to as thft "ppst-^se" questionnaire, even though 



most participants continued thiir 



use of SIES for some time after 



completing^ it. Many variables can plus be examined through changes 
' in responses to items repeated on the questionnaires at two points in 
time. The questionnaires and ma^gihals for responses are included 



in the Appendix. 



\ 



2. EIES monitor statisticsl /on amount and type of use. These have 

■ • V 

been obtained monthly, with the cumulative totals achieved at several 
points in time incorporated into *the questionnaire ( data file for 
cross- tabulation. 

; 4 r 

3. Participant observation. Transcripts have fcreen collected of more 
than 120 conferences, Including some with over 1000 entries. These 
qualitative observations have been useful in providing ,*an 
understanding of what the user groups actually did on the system. 
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•The role played might be described 'as '^observer as participant; 1 ' the 
scientists knew that the author was observing for i evaluational 
purposes. As a form of reciprocity,' £he observer offered tq be of 
assistance whenever /'possible. 4 A passive role was playedi with 



■ / 

comments entered bjr the participant * observer generally* only in 

♦response to a direct \ request for information or an opinion. Many 

participants also shared. their reactions to the system in private 

message* ■ to 'the valuator, and played - the role of informant, 

j • . * 

describing or calling attention to. activities and exchanges 'on the 

system which they thought would be, erf interest In 4 the study. 

* m 

In addition, unstructured face-to-face interviews were conducted 
whenever possible with the principa^investigators^ and/or e^aluators 

* of^the groups being studied*. . ' - 

• ,> 

4 

4 

4. A routine was adapted to .enable ElES to .administer and tabulate 
"the responses to short on-line questionnaires.. This is reported on 
in an article in the ,Winter 1979 issue of phe Public Opinion 
Quarterly. In addition, the system was used to provide reminders and 

• thank you ^ notfes to -respondents to the mailed questionnaires, 
examples % of these are included in the Appendix. * 



5. Since users are requested to send a message to "Help 11 <the user 

» 

consultants on' EIES) about any problem they have with the system' 
a file- was created which logged all these requests^. This file was 
analyzed every one to three months, and served as a basis for mUch*of 
the' "formative" evaluation whicrh provided feedback to the designers 

V 

about aspects of the system which could be improved, on the b&sis of 
user experience. Examples of the Msts of problems generated through 
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* / i- * 

^£he user' consultants lie are included 1A the Append!^ * ^ 

It shouid be noted that the user consultants were Instructed to 
femove any material of a 'personal or private nature and to file only 
questions and consents received tfl&t related to the systenr. SIES 
members were also Informed that their questions to f user consult^its 
wouid be ^ored in** a central file. < # 

b.^A rile was created of thai "^ho-to-whom? matrix - of private 
messages sent, aggregated weekly. This ^ was done for the first 
eighteen months* The resulting data can be used to study \the 
evofution over time of "the network" of social relationships., ^/he 
confidentiality of* such data Is protected by removing the 

identifying* Information through a computer rbutine which substitutes 

■ * ■ \ * * 

a ^random number for the "real" user ~u* *'or this study, only some 

preliminary analysis of changes* in -t*e size .and density of the 
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Communication networks over; time* will be done* The file' will th6n be 



ade available ^to other ^network analysts for more complete social 
neQork /analysis*. This social network analysis will be the subject 
of a: separate paper, 1 to be co-aurthoredfcjith Ronald Rice. 

_ ' t Methodological Difficulties , * ' 

A long term panel study always has problems witft "mortality" amon^ 



the resppnder\ts . In addition, the neesis of evaluators always tena to 
conflict with the priorities and needs, of the organization/sy stem 
being studied. JBoth of these jtroblerns affiected this^ study* 



In ^ addition to tije usual, problem of respondent jroftall&y becausfe of a- 



'decline in interest or moving, this study was plagUed with very high 

• * 
turnover' in ElhS membership during the eighteen months. The research 

design pictured about li>0 EIES members beginning use of ihe system in 

- v a particular month, and continuing to use the system" for eighteen .to 

twenty-four months* In fact, users straggled onto the system. For 
• * 

^y^N^nst^nce, some, CQre members of groups 30 and 35 be*gan use of EIES in 
September/ 1977 • Groups 35, and **5 were to beginln January, 1978, 
but problems with delivery of terminals, user materials, paper, etc. 
meant that .many • did not actually sign, on until the end of February. 

*A*s a result, it was*not possible to find a date^ when a gf*oup could Be 
said to have used the system for three months, which * was t 

, originally planned target for the first follow-up questionnaire. T 

„ kcirst follow-up was thus sent and completed sometime between thre 

■ • 

and six ^months after the date" of first sign-on to EXES. 4 

5>- 



4 



.A more Severe *problerp was turnover of users. Some of "this was done 



^informally— - a person simply gave their* 1 ID to somebody else,, and the 
SlaS staff was inrornved later if At all. Some of it was done 

^purposefully by group leaders, who weeded out inactive members and 
replaced them with new prospective members. In tlfe_ latter case the 



notification of such deletion and replacement .frequently didlnQt 

\ 

filter through to tiie evaluation team and resulted in the absence of 



a pre-use questionnaire being sent at the proper time. Thus, the 

*> • 

number or pre-use questionnaires is the lowest of all. If a person 

% was deleted _from EISS, they might complete a short follow-up 



questionnaire, .but they were not eligible for a post use 
questionnaire, since they were not on the system for a long enough 

4 

time. The end result of ^bhese problems is that the number 'of persons 

Vfiv whom we have a complete, thrfee questionnaire longitudinal record 
i 
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is much smaller than the number for whom we have any one 

questionnaire. A second result of the rotating bodies through the 

* • 
same- ID's is that very careful watch had to be made to modify the 

EIES ID when assigning questionnaire ^ ID's. For instance, assuming 

there was more than one person with the ID of 300, then tj^e first one_ 

was labelled IjOO, the second 2300, the third 3300, etc* There *re 

■ ■ 

undoubtedly a few errors v/here the "wrong" person T s questionnaires 

» > 

and monitor data are being matched for particular points In time, 
although we spent a great deal of time trying to clean the data of 
such errors. A related problem is that "3-6 months" and "18 months" 
are very rough descriptors for the time of data collection/ A few 
respondent's had been ^c^ive somewhat longer than the target figures 
at the time"*" they completed a questionnaire, and ftany for a shorter, 
period of time. 



Further difficulties were encountered with the monitor* data." 
Detailed specifications were worked out focw the type of data and 



automatic statistical analyses that were desired. However, the 

person responsible for .this simply <Tid no,t do the job. As a result, 

* t 

only rainy lidktea monitor data are available, and all 'sflmmary and 

analysis had to be done by hand. 

^ * 

> 

Still another source or difficulty was that the user groups obtained 

an extension of six months to a year in their use of the EISS system. 
■ ■ 

Meanwhile, th$s study had been budgeted far oj&Jy eighteen months. I/i 
fact, the most addicted or committed -members are still 'on line (as of 
* Summer, 1980J , more than six months after the offi-cial end of the 
operational trials. They managed to find the finding Somewhere. 
This totally destroyed the plans for a potentially interesting study 
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of "ex-addicts." The plan had been *to p study the .nineteen heavy users 



In this study who were to lose their memberships. The addicts, 
f 

rt ex-addicts rt and the study could not be conducted 



r 

following their own nfeeds and priorities, > refused to become 



- i 
r 

The most severe of the, methodological problems, of course, is the 
problem of "going native." In order to understand the use of EIES and 
Its evolving electronically based * social system, ' and to remain Jjp. 
commtini cation with the subjects, It was necessary to spend a great 
deal of tine^on line. More than, 3000 hours on, line have been logged 

in the course of , th£s «study. Thtfs, objectivity afforded ■ by 

\ 

"outsider" status- was iopg aga*Iost, 

'V ;■ * ^ 

The'ma^n ^^ution to th^ "going native" problfcft is that the data 
presen^e^ art(K^teEy>*i^£^t^ons made stay as closely as possible to 
^ ob J-ectS&e evidence /^f^plied by the participants themsely&3 — monitor 
data on the £&ouri£ 0 f use, questionnaire responses, excerpts from 
conferences an# messages on EIES* In other words, this report tries 
to summarize stfiat the 'Abjective data, say, and to minimize as much as 
possible any acquired brases ,of the participant^ observer. , * 

. '4 / ^ 
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I Methodological Weaknesses of th* EIES Field Trials ^ 

1 ' ■ ■ 5 -•• • .. 

The EIES project was a unique approach to studying factors relating 
.to the organization and productivity of scientific specialties ? 
actually changing the communication modes of several specialties, and 
tfefen figuratively sitting, inside the v communications network to 
observe what happens. « It is recognizecl\ however, that this fie^d 
experiment distorts and fails to measure what night actually occur 
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should computerized conferencing become a "normal" widespread, 
nonexperimental raofi.e of communication. m j 

1. A New Technology is Limited to a Pew Groups 



One 'analogy that might be made is to the situation when telephones 

were 'hew and owned by only a few persons. Just as people used to 

<; 

have to shout to be heard over long distance and much static was 

■*««. 

commonplace, a f£w technological kinks in the system, which may 
discourage and frustrate users, can be expected in the beginning. 



Secondly, the scientist-users ^*ad to resort to other communication 
Ixodes for other roles they play and their associated communications. 
Eventually., terminals in' the home and the use of computerized 
com erenc^ng might become as cheap % and widespread as 77 ownership 13 
at present. . At that point, people could, belong to many 
"conf erences, " ^corresponding to all their rol6s — a family news 
conference, for example, and a chess conference. For the duration of 
the field trials, however, -djhly the approximately 300 scientists on 
the system were . able tfo be reached by computerized conferencing. As 
a result, use of the system was added on t o— use *of other 
communication mod^s rather > than replacing much of tfreir use. A 
related factor is that for system planning purposes, the specialty- 
group's ability to expand to. include new members^on the system was 
arbitrarily limited. If computerized conferencing were a generally 
available service like the telephone, 1 any number of additional 
persons might Join the network. Still another factor related to the 
newness and scarcity of the technology is fct^at many of £he scientists 
never before used a computer terminal and might not have had any 

31 
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other use for it; tljus* the investment of time tOtlearn the system 1 

might* be problematical* , Since users did not generally have a 

* 

terminal both at horae'and in the office, they had to take the trouble** 
to ^carry it around if it was to be available at all times"* If the 
day ever comes when terminals are as omnipresent as TV's, they will 
always be conveniently at hand without f oreplannirfg, and used with as 
much frequency and ease as more familiar household appliances are 
now* 

2* The Hawthorne Effect * . 

r 

The scientists in this study knew that they were being observed. 
They als.o knew, from questionnaires they received and announcements 
•of. the project, what variables were being watched. This awareness 
may have affected their behavior* They may have been frelf -conscious 
about: what was entered into the system, knowing that "big brother" 
evaluator was out there somewhere reading the transcript* Th$y my 
have deliberately distorted their questionnaire responses to tell, the 
evaluator what they think she wanted to hear* - 

3. Long Term Streets - * . 

\ 

t 

In current experimental situations, scientific groups are only given" 

access to EISS or other computerized conferencing systems for a year 

or two* However, the development of a new scientific concept or the 

f 

transition from hypothesis to proven "fact" may stretch over time 
frames of a decade or more. In addition, short-term recognition of 
, the value of a contribution tends to be conferred by peers within an 
invisible college, but long-term recognition is more liKely to be 



q determined" by users from outside the sub-speciality. 

ERIC a ' 32 46 . 



PARTICIPANTS IN THE STUDY: 
GROUP AND INDIVIDUAL CHARACTERISTICS 



1 



Pour scientific communities (which became groups 30, 35, 40, f -and* 45) 
began using EIES in the period between October, 1977* and February, 
1978, All four agreed to take part in this study. Several other 
groups later joined the system and participated In either the full 
study or in use of sonxe of the same survey questions. 

Group 3*0, "Futures Research Methodology , " is composed of persons who 
have conducted planning, forecasting and similar ^ studies , and are 
attempting to discuss and improve methodology "in this area. As 
pointed out in the proposal submitted for this operational trial, 
"Since futures research methodologists come from a wide variety of 
backgrounds and disciplines, the channels of communication which 

» 

would ordinarily be provided by a single professional society do not 

exist tMartino, S ^977:2). It was hypothesized that use o£ EIES would 

t , 
significantly enhance the rate of innovation^ ^nd dissemination of 

fruitful new ideas in trfe field. These conditions and hopes are 

similar to those stated in the proposals for groups 3b, 40, *and 45- 
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Group 35 Is the "Social Networks Community," which Is composed of 
sociologists, anthropologists, political scientists, and others who 
share an interest in the study of social networks, or the patterns 
and types of "ties" that .connect members of groups or communities of 
various types. As they state in their self description, their "aim 
is to ennance individual productivity and to facilitate the 
development of group goals, standards and the like." 



Group 40 is "tieneral System Theory." As their principal investigator 

states, '"General System theorists constitute one of theJ few research 

communities that are deliberately trying to integrate a wide variety 

of scientific disciplines* The group plans to use the test facility 
i ^ 

- _w ■ 

not only to conduct research, but also to e3ucat,e each other in the 

various disciplines and approaches involved. 'As "common tasks, 11 the 

pactic^p ants yllll compiler* a glossary of terms and a 'disciplinary 

matrix' foe the field" (Urnpleby, 1977:i). 

» « 

■ 

3roup 45 consists of people who share an interest in the development 
of assistive and adaptive devices for the disabled, and includes 
disabled persons, research engineers, and consumer-oriented 
organizations* . V * 1 * 
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3roup 54 is the fifth group which fully participated in the study. 

m f% 

"Mental workload 11 joined EIES a year after the initial "operational 

trials" groUps. They are concerned rfith complex man-machine systems, 

such as the j cockpit of a Jet plane* or the control system in a nuclear 

power plant* One of their objectives was to publish an "electronic 

I 

Journal*" They experienced many difficulties, including the fact 

that a large portion of their group was British, and the 3ritish 

?jr&T (Post, Telephone and Telegraph) *ould not allow them to use 

EIES, even though funds had been provided by the British equivalent 

of NSF. (See Turoff and Hiltz, 1980, for an ' account of this and 

other "electronic Journals" <5rT EIES* Allttse of computer mediated 

communication systems on TbLfc.NET which - might be, used for 

•J 

cross-Atlantic message 'traffic were denied permission by the ?TT, 
which has a monopoly on telecommunications. ) 
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Also Included In the follow-up study "Is a, smaller group (50) which 

used * the system for about three months. It consisted of about a 

dozerf computer , scientists and information analysts interested in the 

use of systems like, SIES for information analysis tasks. An 

interesting aspect or Ciroup 50'i-s that t.hey "moved" to EIES^rom the 

PLANET conferencing system. - t 

■ 

Partial data is available for 3 roup SO, the Hepatitis Knowledge Base 

project of the National Library of Medicine.' 'This group^. included 

approximately ten medicaal doctors who are experts on the disease. 

i 

They used EXES to review and update, a synthesis or "knowledge base'* 
of research results related to the diagnosis and treatment of viral 
heoatitis. 



m addition to . the groups (included in tnis study, many other groups 

f 

used EISS and some ^included an evaluation ^ef f ort which made use of 
'questionnaire it etas borrowed . from the instruments used *on the above 
groups. The data for these groups is not included here, since they 
are not scientific communities* but ma^ be encountered ~ in other 
reports on^the, i.iS,S system: 



-.EG1TECK, a network of state legislative science advisors and 
> \ ^ - * 

resource fcer^ns ( evaluated by Valarie Lamont). 

JEDEC, a standard,s-sett^Q^T^group of , the Electronic Industries 
Association (Evaluated by Peter and Trudy Johns on-Lenz ) . 



» " WHCUS (White House Conference on Library and Information S£rM/lTces J . 

o" - / 35 ; 
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The national advisory group us-ed EIES to-'work with the central staff 
in Washington to plan the conference I evaluated by Elaine Kerr), 



i • . ■ 

The above groups permitted a participant observation in their, 

■ I ♦ 

conferences and activities, and these qualitative observations have 
helped to form some of the Conclusions and interpretations in this 
report. ^ v 

■ * 

"User group *■ is a complex variable which includes _ differences in the 
following attributes: 

■ 

1) "Mature of the task ^ . 

2) Size and social organization of the on-line research community* 
This can influence the amount of information flow. 



3) Leadership style tor in some cases, lack of any leadership for at 

4 

least some periods)* > \ 



4 ) Special software features w£ich were built for some groups but not 



< for others. 



It was hypothesized that group membership would affect perceptions of 
.the a EIES system, and mediate some of the hypothesized JLmpacts of its 



use. 



t 

Characteristics of the Suojects 



< 

Information from the pre-use questionnaire supplies us with a general 

v . ■ ^ 
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picture of the scientists included in T this stutfy. For complete 

percentage destributions on the characteristics summarized below, see 

/ 

* • 14 * 

the questionnaire in* the appendix. ^ 

4 

. " ' • ' r 

In terms of employer, 80£ of the subjects in this study worked in 
academic institutions , 10 % were employed by private research 
organizations, and only a handful worked in business or government 
organizations." Geographically y the EIES users *ere spread throughout 
the United $tates, but ffhe largest concentrations were in the 
Northeast, tficldl$ Atlantic, (including Washington, - D.C, ) and the 
West. A few were located in Canada or Europe. — * 

Almost' all of the^ subjects were males. Most were between 25 and 44 

years old ?nd had a Ph.D. ' They tend to be "mid career," having 

received their degrees five to nineteen years previously. A third 

were in the midst of writing one or more books when they joined EIES, 

and the Majority were working on one or more Journal articles;^ 

Almost air had published one or more Journal articl^^previously, and 

about a fifth had oublished thirty or more articles. .Compared to the 

.total population of scientists, then, most of whom have never 

published anything, the scientists using EIES were considerably^ore 

*» «• * 

\ 

productive than average. They are hard working, with the majority 
repotting considerably more than the forty hours which most Americans 
think of as a rf normal" working week. Much of their time is spent 
teaching, reading professional literature, doing research and 
writing, with meetings and administrative duties taking considerable 

time for some. 

w • i 

■ 

Although most were not previous users of computerized 



communications system, they had used, computers and computer terminals 
before, and had positive attitudes toward computers. 

■ 

Terminal access was less than ideal. Only about a quarter had their 
, own terminal in their own office. One in five reported no regular 
terminal access at all. Thg majority did not have.a terminal which 
they could use at home. ■ - - — 



r • 
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CftAPTER TWO 
DETERMINANTS OP USE OP. THE EIES SYSTEM 



One of the most intriguing aspects of computer-based communications 

* * % 

% 

systems is the contrast between users who integrate 1 t^is new form^ of 
communication and , inf o^mation exchange, into their lives and those who 
do not use it at all, even if they have free access. What explains 
or predicts acceptance of a system such as EIES? 

In this chapter, we will look at which- variables do or do not explain 

I 

differences m in amount of use of the EIES system. The other aspect of 
user acceptance, subjective opihions of the system, will be examined 
in a later cfrapteK By way of introduction and summary, it may be 
saicf that the various pieces of data all point / to one overall 
conclusion; It is aspects of the subjective motivations of the 
participants, not the objective cnaracteristics of the system*, which 
are the primary determinants of amount of use, at least in ^ terms of 
Initial system acceptance. This is not to say that long-term users 
are not sensitive to the objective characteristics of the system oj* 
that system characteristics do 

statisf action, the choice among available systems, or the range of 

* * . « ' 

professional activities for which a Computer-mediated system will be x 
used.- 



not influence - subjective 
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' CONCEPTUAL FRAMEWORK 

i * 

■ w • 

Expanding and Building upon the original list of factors generated b^/ 

Vallee et -al. (1974:22) one can categorize the determinants of*- 

acceptance and use of computer-based communications systems as 

determirfed by characteristics of the INDIVIDUAL - USER, . the SOCIAL 

1 

JROUP OR ORGANIZATIONAL CONTEXT, the TASK, the SYSTEM it-self, and the 
\ . ' - • 

EQUIPMENT which the individual and group have to use *ith, the 
"system.* These sets of factors may be treated as competing hypotheses 
or alternative explanations for predicting^ amount of 14s e of the 
system. 

• * 
The full frameworks for potentially predictive characteristics of the 
individual and of the social groups or % organization are shown in 
tables 2-1 and 2-2% Within the context of studying only five.SIES 
groups (whfch did not have any particular task and which were 
confined "to .a single system, with little variability in available 
equipment), most of the attributes of Task, System and Equipment that 
"have been developed could not be included in this study or 

* - V ' . 

determinants of system use* 



•This framework was expanded and developed ir* a workshop project 
funded by the Division ♦ of Information Science and Technology, NSF. 
Contributions were i*ade by" Mutfay Turoff, Valerie Lamont, Elliot 
Siegel and John Senders, as weft as the author of this report, who 
Simultaneously P.I. for* the workshop project. 



o 
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In re*g*rd t'o 'the- SYSTEM and EQUIPMENT, we have some data on the'' 

JtetAjaffect^ of the following: . 

DOCUMENTATION (was It clear and -comprehensive, or not) . fc. 

Whether or/not there was a "live" teacher provided 

•\3uality of the "TELENET interface (whether or' not TELENET was a source 
. -Of "trouble") .' ' ' , . fi 

Whether or not the 3ystem was a source of difficulties /^\. 

System* Availability (downtime during workday or unavailability lights 
and weekends) - r * - V* 

rouble with- Jrhe telephone "or. high -cost of 'long distance telephone 
ue to absence of. TELENET node ' "~ ' 

-rt Access to. terminal (own-or share or none at, office; own. or Svairaode 
^ lQan or % no'ne at horrte) * » 1 \ 

CRT, print, oc bot^h * , " , , * . 



Size and weight and printing speed of the. terminal(s) avaiiaffe 



rfe have two sources of independent variables in exploring the source 

,of determinants of the amount of* use of EIES* .The first consists of 

/ , * ■ ' ' ■ i - 

data front the fbllo^-yp questionnaire in >which t*e» respondents 

themselves report what factors are important in limiting their. use of 

EXES* Then we will turn to d^ata on variables included in the pre-use 

questionnaire and examine correlations between initial attributes of 
* * ■ * * 

the individuals and the amount of use which they make of the system t 

\. / 
There are also a few. other variables measured on the follow-up 

* +■ a * 

quest^r«ai r re»whj.ph may Help to explain variations in use, such as 
w v - . ! ' 

leadership behavi6r» * 

* »" • / . . * 

* & * ' ^ 

* " . ■ 

Correlation apd significance statistics, will bemused to categorize 
^served relationships as v stronffljiod^rate, weak, or* non-existent** 



4 





*When examining correlat ipns , the j^st frequent measure will be 
gamma, which is appropriate fpr linearly related opdinal variables. 1 m 
Occasionally, the pattern of correlation Is curvilinear, in which 
cases we will reportf ETA, a measupe of VcurWilinear correlation.- 

Chi square tests. are used fop all cross tabulations to estimate the 
signif icanoe of th* pattepns of association. The results of the chi 
squape tests should be intepppeted as a vepy pough measupe of the* 
extent to which the number of obsepvations and the pattepns. of 
association observed ape large enough to sepve as the basis fop 
gerferralizable conclusions. Since the pespondents do not peppesent a 
raniom sample of all users of EIES, let alone. of 
of (all such systems, ^chi squape op t-test 
intepppeted rJ^goroitsly 'in tepms of a level' 
generalizing to such a lapgep population. 



all potential useps 
results cannot be 
of* confidence" ir) 



In looking at correlations of ppe-use atti 
with subsequent houps "on line, we "vlll pefep 
to* .2SU accompanied by probability levels 
"weak" relationships.. If^the correlation is 
significance tests indicate that- the prob 
could be^tccounted fop -by Sampling error is 
say tfiat. thepe is "no relationship'^ Modepa 
cop/elatiops between *20 and .49, with at 
significance. "Strong" relationships will b 
that are characterized by . cofrrjelation coeffi 
significant at the .05 level or better. 

In looking at directly "reported reasons expl 
system, we' will call those- named by 20? 
"moderate"; 5-935 "weak";and less than 55, not 



tudes and charact 
to correlations 
of .10 or less, 
les« than rlO 
ability that the 
greater^than .20, 
te relationships 
leas,tf a .10 le 
e said * to exist f 
cients of .50 or 



eristics 
of '.10 
as being 
or the, 
results 
we will 
refer to 
vel for 
or* those 
greater, 



aining limited use of the 
more "strong" \ 1*0-1955 
a determinant. o 
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• TABLE 2-1 

CHARACTERISTICS OP INDIVIDUALS WHICH HAY APPECT SYSTEM ACCEPTANCE 
* Indicates that one or more measures of this* factor were 
included in this study * - 

.A. Attitudinal variables 

1. Attitudes toward task 

a) Reiative importance or priority* 
bX Degree of liking or disliking, of the task N 
(pleasant/unpleasant, chall$nging/bori-ng, etc.) 

2. Attitudes* toward media ' * V 

a) Attitudes towards computers in general* 

b) Expectations about the specific system 

1) Anticipated-u£ef ulness (amoutjt of use)* 

2) Anticipated impacts on productivity* 

3) Anticipated difficulty of, use 

c) Attitudes toward alternative media (telephones, waiting 
letters, travel, etc.) 

3- Attitudes toward the group (liking, respect, whether they are 
an important reference group) 

4. Expectations about how system use will affect relationships 
with the group* 

5* Perceived pressure to use the system* 

3. Wor& Related Skills and Characteristics * 

1. Personal communication skills 

a) Reading sp.eed* 

b) Typing speed* # 

c) Preference "fop speaking ot^ writing* 

d) General literacy (writing ability) 

2. Previous related experience^ 
a) Experience using computers* 

- b) Use of computer terminals* 

cj^lfse of ofcher computer based communication systems* 
* 3.* Physical or intellectual disabilities 
Productivity 

a) Hours pec -week worked* 

b) Number of publications or other output measures* 

5* Connectivity - > „ 

Number of persons in field with whom one is in contact* 

b) Number, of persons on system with whom one was in previous 
conta'ct* . 

c) How well known person is in field* - 1 

d) Whether a scientist -Xeels "in the mainstream" or nqt* 

e) Number of coauthors (or coworkers-)* 

C. Demographic characteristics . 

1. Age* - 

2. Sex* > ' 
' 3* Educational level* 

4. Race, nationality or 'subculture 4* 

D. environmental variables 

1. Available resources, including secretarial support 



\ 



*• 2. Position In the organization Cor status in" Informal group)* 
3. Amount of pressure to use the system (from superiors and 
peers)* * 4 " 

]■ £. Psychological variables # - ^ 
I ' 1: Personality characteristics 

K (e.g. Xtroversion vs. extroversion, as measured by Myers Brlggs 
type, Indicator) 

2. Basic values (e.g. the pattern variables - universalis^ vs. 
particularism; affectivlty vs. affective neutrality)* 



\ 
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. ; - TABf-E 2-2 * 

- GROUP. FACTORS WHICH MAY APPECT SYSTEM USE 1 M 

•Indicates' that f a measure of this factor was Included In this 
study , * 

-A* STRUCTURE^ ' . . 

1. Size* 

2. Degree of geographic dispersion 

3* Centralized vs. decentralized control 

4. Pre-existing communications ties or network 

3. LEADERSHIP 

1. Style 

*.2. Level pf effort or activity b^htf^leader 

,C. C0HES1VENESS , . * - 

.1. Socio-raetric ties' 

a) Have they met face to face? 

b) How many members of the group are known to each other before 
they begin communicating on £he system?^^ / *; 

a) Have they worked together previously?' 

m D) do they form cliques, have^wany "individualists/ 1 or ape they 
an integrated group?* * 

2. Competitiveness* 

3» Trust or openness ajnong members* 

ij. Status Care most group members prestigous in their fields, or 
not?)* 
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Measuring Level of EIES Use 

» * 

» 

An overall profile of the "average" (mean) use of EISS during the 

operational* triaJLs is shown in Table 2-3. This is derived from 

monitor data on the cumulative activity of all EIES members as of 
v * 

April 1, 1980. At that pdint in time, more than half of the members 
of the system were non-scientific users,, and some of the members of 
the operational trials groups had been gleleted and thus are not 
included in the computation of the average. The data do give us a 
rough idea of the usage patterns of members. For -Instance, rfe *see 
that users did most of .their seoding in the fprm $f private^messages, 
which go to abo'ut two persons on the .average; but most of their 
reading in the conferences, *where, items are read by about twelve 
persons, on the average. We al^o note a fairly long average session 
length (24 minutes). 



However, usage is nighly skewed. ?able 2-4 shows tne* somewhat 
astounding fact that 40? qf the scientists invited" to have free 
access to EIES 1 " either never signed on at all, or dropped out before 
learning to use the system. Within this* "dropout" category, 112 of 
the sample never signed on at all. 




In a system such as EIES, when use is voluntary fdF-irtfst member>s 

(such as during the operational trials), amount* of use in terms of 

hours on line, can be taken as a fairly valid measure of user 

acceptance. However, lack of use in the totally— ^voluntary, " almost 

"extra-currl^cular" mode that characterized the dpe^ational trials 

cannot be assumed to validly indicate rejection of the system. It 
* 

ERIC . . . ' . 



simply indicates that the relative costs and benefits were mpre 
favorable for off-line activities? (In other words, # low use hag, to 
be accompanied by poor opinions of the system in order to indicate 
active /'rejection." of tfie> system. ) f 

4 

Since use was skewed and our independent variables are mostly nominal 
or ordinal, cumulative hours on line has been divided into levels or 
categories for most analyses.' TJUs procedure has tne advantage of - ^ 
not weighting the small number of users with very high numbers of 
nours of use too heavily in the analysis. It has much the same 
analytical effect as using the log of the number of hours, in those 
analyses where both methods of handling the dependent variable were 
tried and compared. 

The first level consists of tnose who did not accept^ tne system: 
never signed on at all, or did not stay on line long enough tp get 
tnrou&h the learning period and be able^ to use the system effectively 
(this is less tnan five hours total use, referred to as the ^rop 
puts"). "Low" use level is 5 to 19 tvburs, "intermediate", 20 to 49 
hours; "high" use 50 to 99 hours on line; and "very high" is more 
than one hundred hours of 'connect time. These break points 
correspond to observed changes fn user behavior derived "from monitor 
and questionnaire data, as well *as giving us reasonable marginal 
distributions among the levels. 

These data are available for cumulative hours on line at follow up,, 
post use, and several other points in time. The follow-up data have 
been chosen as the focus for this analysis. One reason is tnat this 

is the point for which we have the moss questionnaire data. Even the 

' # 

6'/ 



"dropouts" ^ were sent a two page follow up, asking for a rankfog of 
reasons for not using* the system* Tot air responses to the short (for 
dropouts) and long follow f up questionnaires were 195 out of 213 
members of the groups in the study; ^ almost tyice as much 
questionnaire data as are available if the post-use questionnaire 
were used. Another reason is that "acceptance" or "rejection" can fre 
fairly clearly established in the first three to six.aontns. If a 



... 

person does not use the -system in that period of time, tAey are very 
unlikely to ever use it. In fact, many of the non-use?s were 
subsequently 'dropped from the system by the group leaders. 

r # 

Table 2-*?£ndicates that usage patterns were correlated with group. 

3coup 5< (Mental Workload; nad the hignest dropout rate ^ 625; • Many 

of these w^re the 3ritish users who were refused access by the 

3ritish Post Office. 3roup 45 ( Devices) also had a large number of 

invited ^participants who never became active. The lowest dropout 

rates were among, the two task-oriented groups (50, Information 

Science, and SO,* Hepatitis). These also happened to be the smallest 

groups, thus, if there is any overall relationship between group size 

and amount of use of a system by its members, it cannot be determined 

from the operational trials groups— We will look at a few 

group-related variables which -seem to predict amount of use of 2IZS 

in this chapter, related to the PERCEPTIONS of the members about^h^e 

competitiveness or unethical behavior of the members and total 

\ . 
Self -perceived status level of the group's members. Other g£*oup 

factors which may explain these variations are further ^explored in 

Chapter Three. . ** 
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CATEGORY 
Hours Used 
Number of Sessions 
Average Session Time (minuses) 
Text Items Composed 
Text Items Received ' 
Items Transacted/Session 
Average "Input Rate v 
(words/minute) 



TABLE 2-3 . 
AVERAGE USER PROFILE 

AMOUNT 



SUBSYSTEM * 0? ITEMS % OP ITEMS 

' COMPOSED RECEIVED 



105-5 
265 
. 2^ 
279 

1,19* 
5.6 
7-9 



SIZE CIRCULATION ' 
(LINES) RATIO 



Messages 



69. 1 



35. a 



10 



2:2 



Conferences • 



22.3 



60. 9 



14 



11.7 



Notebooks 



8.6 3.3 
r 



/ 



19 



1.6 



Source: Accumulated Monitor Statistics as of April 1 1980 
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Group 



TABLE 2-4. 
iOURS JON LINE AT FOLLOW UP, BY GROUP 

less than 5* 5-19 20-49 



50+ 



30(N-35)< 

35<N«40) 

40(N«5U 



345 



325 



335 



20* 



255 



335 



295 



225 



.175 



155 



125 



45(N«4«) 
50(N«ti) 



.5^5 



125 



255 



625 



135 



135 



45 



1"25 



54(N-21) 



295 



55 



80CN-10,) 



205 



505 



105 



Total(N»213) 



405 



295 



205 



115 



•Includes persons who never signed on 



Source: Monitor statistics for cumulative time on line, June 1, 1978 

p 

i • 

« '- or beginning of month when follow up was returned 
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SUBJECTIVE REPORTS OP FACTORS WHICH LIMIT USE OP EIES/ 

■ ■ 

Table 2-5^ shows, the overall ratings of the importance of various 

• « ^ 

factors in limiting use of EIES, at follow up. These responses lump 

together the dropouts, the very heavy users and all those in betweert* 

A subsequent set of tables, at the end 1 of this chapter, breaks down 

the responses^by level of use. Surprisingly, there are not many 

differences by -level :^ reasons given *as very important by those who 

never— ^sed the system or used it ~very little are almost the same in 

terms of frequency of mentions as those given by heavier users. The 

main results of these cross tabulations can be discerned from the 

correlation coefficients reported in table 2-5* in conjunction with 

the results r of the Chi > square test which indicates the level of 
■ * > 

• - 

statistical significance of the observed correlation. A minus sign 

1 in front of the* correlation coefficient means .that the reason was 

given more frequently by dropouts and low level users than by * high - y 

leveKusers. 



The reasons in table 2-5 have been listed in order of the frequency 
with Khicn they were named as "very important" by, all users, with 
"some weight given to the frequency of "somewhat important" responses, 
We see that conflicting demands and priorities are by far the most 
important barriers to use. One reads *the data in Table 2-5 as "* 
follows. Overall, 47* ^of users report that. an important limitation 
on ' their use is that "other professional activities must t^xe JUgher t 
priority."' The frequency with which this Yeason is indicated is 
somewhat higher for the dropouts and infrequent* users, indicated by a 
Gamma of -.17. ^his weak relationship with hours on line is not 4 

• • ' -.i ■ . 
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statistically significant Cp«* 16) * The full tables from which the 
gamma £nd p figures are drawn'are in the appendix to this chapter. 

Qualitative data from ' the post-use questionnaire reinforces the 

H • 

importance of^ the relative priority- of the task in determining level 
of use of trie system. Many respondents indicate in their open-ended 
comments that the rfork for which they are being paid conflicts with 
use of EIES* In fact, many see EIES as taking away from the time 

"needed to do their official Job* Communication with ones peers in 
othe$ institutions ^ is^ simply not as high a priority as the work 
commitments pressing in at the workplace. A selection of sucn 
comments, from the open-ended questij^on j;he post-use questionnaire, 
is shown in Table 2-6* (The ful^Cist appears In the Appendix). 
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A related .motivational variable i3 having "no one on the system with 

whom one wishes to communicate a great deal". Though only 7% of all 

EIES respondents list this anti-social sounding* reason as "very 

important' 1 , those who do feel this way are likely to be dropouts. 

JIoC , a single user in the sample who did not particularly* want to 
* • \ * 

communicate with the limited community oh line logged over 30 hours, 
and the correlation (gamma* f -40) is the strongest for any of the 
self-reported reasons for non-use. 

After the motivational* variables of conflicting priorities^^fEf lack 

of desired communication partners, but far behind, are favors that 
, nav^ to do with" access ^to the system* "Limited night or evening 

hdurs" was a strong enough deterrent so that steps were taken to put 
^EIES up seven days a week, Ground the clock* During nights and 

weekends, someone is not always at the console in case of a .crash, 



but a system wa^g devised whereby EIES can be restarted remotely, by 

telephone,* if It is found to have crashed. * , A 

■ 

Another access barrier ranking high on the list of factors which 
decrease use of^ SIES is trouble with the TELENET link. (The more 
time they spend on line, the more trouble they have*. And .^EiENET's 
reliability has &een decreasing, not increasing. Se£ the discussion 
below). Closely behind this is the ^elated access barrier of trouble 
with tlfe telephone connection. But note that reporting of all of 
these access barriers INCREASES with' use. •••in other "words, 
encountering access difficulties does not 'cause low use, but is 
rather proportional to the amount of use. 



The one frequently mentioned access b&erierl which does appear to' be a 



cause of low use is inconvenient access to a 



Tr; 



terminal. 



Characteristics of the system — having bad experiences such as a 
crash, or the feeling that i-t is "too complicated" — are "somewhat" 
important reasons cutting down use, but are hot very important to 
many users. "3ad experiences" ueak3* in the ^ow use r;ange (5-19 
hours), where 40J5 say this ha^been ^somewhat " important in cutting 
dowp use, \ . 

The relatively low-^ pronrinance given J^o cost is probably attributable 
tci t£ie subsidized memberships of the users. , They generally had to 
pay only local telephone charges to'reaok- a TELENET node. For 
non-subsidized users, cost would undoubtedly - be a more important 
factor accounting fcflr level of use. > 



> .^ v ' Telenet ^roubles 
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In th* Appendix is a selection f com' the first fift5t comments of the 

public cdhferencre on EIES , established to air "TELENET Experiences 

Begun at thte end of the operational trials,, it acquired 72 entries 

the first % month, most of which descriptions^ of difficulties ♦ The 

number of TELENET difficulties encountered during a month by all 

— 

users is undoubtedly many times that>which users take the time and . 
trouble to document in the public conference*" 

. ' . / . 

m ■ * 

$ 

Tbe "norm" on EIES is that* it-ems in public conferences are indeed • 
that: public statements and quotable without permission, This^is nort 
zrnp of group or private conferences, for *hich thet, norm is that 
permission to quote or disseminate further should be requested* The 

selected items have been IncorpQrated intact, complete rfith whatever 

> 

typographical errors or differences in formatting appear in the" 
* original* Aside- from the content, several things should be noted 
about this transcript as an example of a computerized conference: 



L 



1) Generally, the agenda* and groupd rules Ifor di3cus3ion are proposed 
by the moderator, and dl^sfcussed and agreed upon at the beginning of a 



conference* 



2) The comments actually ^do refer to and build upon one another, 
s 



constituting a genuine multilogue rather than a series of discreet 

/ 

monologues* * , . 

■ « 

s > 

3) Pen names are frequently used as a device to play "devil's 
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advdftate." Anonymity Is also frequently used In this manner, and 
J)also y to express feelings* that the' author may* Intend ta represent 



;hose of other members 4 , too. 



The TELENET difficulties encountered included the fallowing: 



I) .Local .TELENET «o^es become overloaded; they simply do not answer 
when dialed or they giv.e a' busy signal. * \ 



2) Orle or more^ local nodes "goes/ out of service. If it is the Newark 
<n no£e, then no one can reach SjES through TELENET. ' 

3 J Users af*e dropped by TELENET- and are- 'Trozen^/'on line'- Mose 
"specifically, somewhere ~Tn the network', the fact that the user is 
^connected to EIES* gets lost. The packet loses Its ad'dFess^ s,o /to 
~speaK, and does not get delivered to the EXES computer. Th^tiser 
^.npurs^ and gets no, response, because EIES receives nothing to respond 
, to* Meanwhile, the port -on EIES .sits open, with EIES malting for the 
- lo»? packets that nev^er arrive. If the user hangs tfp and redials, 
x ' she oi^ he discovers that "That ID is In use." EIES has receded no 
signal that the user hung up the phone, and keeps thjfiftllne .ooen until 
either the automatic time out occurs (for vthich the default is set .at 
twenty uinutesj or a privileged *EIES staff member "bumps" the frozen 

M. * 

ID. ' Users find themselves, if they know someone else's access code> 

* * 

In *tji6 absurd position of signing on as someone elsl^in order .to 

-■request that their own ID be "bumped." Or they cjill Newark. Or they 

Impatiently 'wait for 'twenty minutes'. Or they give up. and end tjie 

session. ; " ,*».' 

' • " ' : , . 
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4) Most seriously of all, TELENET sometimes mixes up packets and 

swi-tches t users, even among different c^hputer systems. ■ (See 

especially, cc33) • '* 

< k 



AJLong with tfnat most users felt was a constant decline in the quality 
of TELENET- services, she faXi.of- 19#0 brought a Vise in price for 
TELENET: from 3-75 ttf $5-00 per hour— a £R'f f iciently .large Increase 
constitute an economic problem for many users supporting tneir own 
h account charges. As one user summed up the situation ('Douglas Cayne, 
in cl011cc35), 

_f the^ networks can do no better than offering this 
, sort * of consistently ' poor — borderline 

unusable — service, it may be many .more years than ' ** 
, ^ *e T ve been predicting, before we become the Network 

Nation; or before- people 'find computers useful enough 
to have in. the home... 



Table 2-5 

- * 

. * Importance of Various Reasons in Limiting Use of EIES^ 

And Correlation (Gamma) with LeveJ. of^rUse > 

Reason- * * * _ very Somewhat Not Im- Gamma 

, , Important Important portfant 

Oth.er professional nj% 30 22- -.17 

activities jfrtist take , 
, higher ppiori.ty 



P 

16 



Limited night or" evening 20S ' -21* 60 27 * 05 

'hours v * 

t . 
Inconvenient access to a 19s ..18 63 --.15 .16 

terminal / 

Trouble with Telenet 155 19, t5 *f^~*l\ "'.01 

Had some bad experiences 112 - 31 ,58 5 .29* .005 

The system is too . 25 "66 .12 .001 

complicated . , 

* # . rouble with telephone 105 17 74 .20 .05 

v 

Cost fcf telephone or '\ 95 11 80 .08 45 

Telenet * * • 

There is no one on this" 75 16 77 - 40 14 

system wl&h whom G wish \ - 

to communicate a/great 

deal " * 

The, conference comments , ( 7% 31 52 wOl .05 

, - or messages I have ' ■ - • 

% 'received !lo not seem . . 

worth reading* * - 

Red notebook ( 27 68 --.13\ \04 

documentation looked ^ t 1 

like too. much to read , ' * ■ 

\ . 

Inadequate leadership of 5£ 17 78 . .59 

* . the group t , * ■ 
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Table .2-5, Con't" 

Reason ■ Very Somewhat Not Im- Gamma p 

% Important Important portant 

I am not very Interested * 65 17 77 -.02 .60 

'in the* subjects being , • * 
discussed 

I do not kndw how to type, 5% 15 80 .08 ' .5H« 

I do not iike using a , 336 * 8 89 - . 15 .76 

cccaputer sys??m like this « ' 

* 

Source 4 : Follow-up questionnaire^ sent to 3roups 30, 35 > 40, 
^5, 40, 5^> so. ' " 

A Total N responding Is 195 
Note: Samma» correlation with accumulated Hours on line at' follow-up, 
'categorized by level, v A rt ralnus" gamma indicates that the less time 
on llne, # - #J6he more likely the person ^was to' name the reason as very 
Important. \ * " 

^p"* probability that the correlation could be attributed *to 
sampling errorr, ba^ed on Chi square test. * s 

■ • 



\ 
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TABLE 2-6 



'What one or two factors best explain why you h^ve not used EIES more?" 



Post-Use Open Responses Emphasizing Priority Conflicts* 

I. I f m very busy, with\heavy commitments* El£S doesn't contribute to 
any of the things m V really MUST do!" It is a Peripheral interest*" " . 

„2. Too busy with other things a 

3i Time pressures resulting from need to EARN by b consulting, and 

teaching extra loads . 

c 

4. Lack of tir\e- other research projects are more pressing 

~ , i \ ^ 

D. Lack of yime and pressure of my business — I en\ associated with a 

small RiD f ^ ,;n * hich implies a constant need to se>ek* new contracts., 
6* I am under a great deal of time pressure " , 

4 

7. I work full time and ^m a full time graduate student and half-time 
mother — need I $ay more? 

7. Other matters, with.mojje immediate DEADLINES, kept interfering. 

5. Very busy with other things such sfs classroom teaching,* talking 
witn students j working on articles and proposals ; committee *rork. 

9* Exttrnal pressures for time keep me elsewhere. Except for a few 
direct' research collaborations over SISS, the rest seems morfe liice an 
interesting luxury'than a* necessity. t . . 

fO. There is -no job related* reward. EIES takes* time and.is not 
recognized by the university ... thiS is unfortunate. 

II. Pressure of administrative responsibilities. 

12. „ It * is extremely difficult to match full-time (university) 
professional ijfterest and responsibilities tfith***fciiose generated by 
the wide memSership of £ISS. 

• * / 

■ * * * * -»w^ + 

13* Extremely busy schedule during last year. 

14% Lack of time to participate* THIS* IS THE ONLY reaspn^ 

^ 

15* Work pressure ■ * 

16. Other time consuming work is more pressing % 

■ - . \ \ 



y * 
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; * The Relationship Between Alhount of Use and* 

Reasons for Non-Use ^ 

Looking at the set of cross-tabulations of reasons limiting u-se of 
EXES by actual hours of use made of the system (in the Appendix to 
this^ chapter) , there are far fewer differences than might be 
expected. ^Higher priority for othe^ (off-line) professional 
activities is particularly impor££nt for the drop-outs (named by 
55%)* The massive nature of tlie^Ved notebopK" documentation shows 
ap most frequently as "somewhat" important for the intermediate-level 
users. Dtrfti the feeling that the system is too complicated, becomes a 
deterrent/ &t the higher use levels, more than the lo^er, somewhat 
surprisingly* 7 

Reasons Siven by Dropouts 

■ . 

*A subgroup of particular interest is the "dropouts". Jhe following 
are the only reasons listed as "very, important" by 105 or more of 
drop'outs: * ^ « 

fttber professional activities m . 55% 
'Terminal access . * 195 ' 

* Limited' night or weekend hours 125 $ 

No one to communicate with 125 # 

Trouble t^h telephone 11? 
, > Material not worth 4 reading t 105 



impc4l 



^Looking only at the reason named^atf the single "mbs>fc irnpdrtant ," 
> t+ 

conflidt in priorities with other professional activities is the -only 



er!c \ ■ ■ ^ 



reason giv*n with great frequency by the "dropouts" < those who never 
w spent more than five hours on EXES). The second most frequently 
U.sted tT most important factor" by the dropouts is inconvenient access 
to\| terminal, named by 9** (The complete table of these data is not 
included here. ^ Almost all reasons, except the above two, are named 
as "most important" by only a small number of people). 

■ i ■ ■ 

-PREDICTOBS FROM THE PHE-USE QUESTIONNAIRE ' 



Many of tne questions in the pre-use questionnaire, Pleasuring 
motivation to use* the system before having any * experience witn it, 
turn out to be significantly correlated' with subsequent amount of 
use. This includes anticipated value of the system (Table 2-7) and 
amount .of time s*pent on" the pre^use Questionnaire CTable 2-8/. The 
latter may* 1 seem to be a, surprising predictor, but it is an 
interesting . benavioral measure' o£ pre-u3e attitude toward tne system 
and the project. The strongest predictor is the 'amount of time which 
a prospective user -estimates that s/ne will s^end on line eacn week 
.Table 2-9). Two thirds of those who felt tnat they would spend less 
than 30 minutes a week on line became dropouts. 

On the other nand, most of the "objective" characteristics of users 

tnat mign t be thougntf td predict acceptance, such as typing speed, 

* * 

did not turn out so be related to amount of use. 

■ • » * 

Estimated number of -sign-ons per week, before the system was used, 
follows the same pattern as anticipated time on-line per week. A 
third ,2b of 39 responding) estimated tnat they would 1 sign on only 
once a <*j£eeic or less. ' Twenty-tnree of these users in fact were drop 
outs or low level users., (g&n^ia- .50, p s .02). 

r 
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/ ANTICIPATED WOftTH OP SYSTEM BEFORE USE, 
BY TOTAL TIME ON LINE, AT FOLLOW UP 



Totjaf 
Hour* 



tical 



Use- Others Sleep- 
less; 

505 



Neu- 
tral 



1005 



05 



505 



05 
10Q5 
2 



05 
1005 
1 



255 



505 
255 
05 
05 



1005 



05 



05 



05 



05 



Lim- 
ited 

365 

f 

' 255 
365 
35 
05 



Use- Revolu- 



ful tlonajry, pends 
255' 05 6*J 



De- ^ 



1005 **100jf 
'2' 2b 



355 

135 
55 



1005 



05- 



335 



2o; 



205 



175 
505 0 
1005 100 



05 



4-0 



r 



Chi square 3 ««.75i*5 p .02 gamma - .27 
' ' ' V 



Question: . 

'Which of the following 3EST describes your anticipation of tire 

system's wortjh?, , 

— I think it will be useless 

think\it is useful for others, but not for tie 
— I am skeptical about it but willing to try it * 
— I am basically indifferent or neutral ** 
—I think it will have some (limited) worth worth for % me 
— I think it will be useful 'in ^many respects ) 
— I think it will revolutionize my work/communication'ajrocesses, 
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76 



62 



1 




5-19 
20-49 

50-99 
100+ 



TABLE 2-8 

TIME 'SPENT' TO COMPLETE THE ?k£-USE QUESTIONNAIRE, 
3Y TOTAL TIME ON LINE," AT FOLLOW UP 



1-10 
Minutes' 



«75 



305 



145 



65 



11-30 



r 



minutes 



335 



385 



85 



05 



21-30 
minutes 



115 



375 
325 
105 
105 



more than 

1 

30 minutes 

■355 
155 



355 



95 



65 



N Responding 136 



24 



19 



34 



Source: Pre-use Questionnaire Question. How l0£g did it take you to 
complete questionnaire'? 

v Chi square 3 22.15, P = .03, gamma=» .30 



* TABLE 2-9* 
ANTICIPATED WEEKLY USAGE OP EIES, BEFORE USE, 

i 

BY TIME ON LINE AT FOLLOW-U? 
t < than 30 min. 30-60min. 1-3 hours 1 hours" 



< than 5 hours 623 * ' ' 35* ^0% . 4 



'5-19 hours 255 50% , 20% , 205 

20-49 hours 13* - 15* 29* 44* 

i 

50+ hours 0* ' 0* 1>* -325 

* f 

N t 8 . 20' «- 35- 25 

100* 100* 100* 100* 



•a 



Chi square's 50.7 p - .001 ' garana = .54 



Source: ?re-Use Questionnaire . 
Questio'n: How much time in the average week do you f oresee\vourself 
using EISS? 



Connectivity 



There is a weak to moderate relationship for various neasures of 
general connectivity to other professionals. For off-line 

relationships, specifically number of coauthors in tne previous year 
and total number of persons in the specialty with whom the member is 
in .contact, the relationships seem somewhat curvilinear. ,That is, the 
isolates* and the socio-metric stars do not use the system as much as 



1 

•3 



those witr. moderate numbers of professional connections, who seem to 
c 

> have thfi most motivation to expand their professional networks. 

* * 

In tJerms of previous contacts with members of the actual on-line 
group, the relationship becomes very strong. The question asked at 
pre-use was"* how many persons among those in the specialty with whom 
the scientist had contacts were in the oroposed EIES group. 
Previously Knowing a large number of the on-line, group faembers is the 
strongest 'pre^ctor of very high levels of subsequent use of the 

system. « A sefies of step-wise multiple regressions was conducted to 

i 

. find che stri>i^est combinations of predictors of amoun^of use of 
EIES (see the fend of* the chapter). When the total number of hours on 
line at folTow up wa^ used as^the dependent variable , ( rather tnan tne 
log of the number * of hours, or the level of use, in categores), the 
strongest predictor * i*s this variable (Pearsonls R * .48). 
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The Effects of Perceived Competition - ' 

\ 

Contrary to the* hypothesis that the higher the perceived level of 
overall competition in t^he specialty, th£ lower the amount of use of 
EIES, there is no significant relationship* With the degree of 
overall competition categorized 'as intense, moderate or weak to 
non-existent, the correlation (gamma) was only .13* and it was 
statistically insignif icaVit ' (p * .72). Ther.e is a stronger 
relationship at the GROUP level (see chapter 3)* 

« 

However, there are relationships between perceptions of specific 
KINDS of competition and amount of use of EXES* Those who perceive 
comptition over funds are slightly more likely to drop out (33* vs, 
265!) t and less likely to become heavy users in the first three to six 
months (9X of those reporting oompetition related to insufficient 
■funds logged fifty or more hours, vs. 18S of tho^« who did not 

perceive competition of* this sort. Overall gamma=.I9, p» .13).- 

\ " 

•.Only seven persons who reported competition related to unethical 
practices among some scientists ^ in the field also completed the 

9 follow-up questionnaire • This makes*, it unlikely > that any 
statistically significant relationships can occur related to reported 
presence of, unethical behavior (which is interpreted*" as & measure of 
trust in the group) However, as we see in Table 2^0, there d^es 
.appear to be a. relationship. Given the strong relationship but the 
low , number of subjects, we will call this relationship "strdng," even 

. though it does not meet the statistical significance guidelines. 



ERIC 



3n the other nand, there is an apparent tendency for those who <feel 
that , competition' in their specialty consists of arguments among those 
. rflth strongly opposing views to spend more time on HIES. Only ISIS of 
those reporting this reason for competition dropped out, vs, 353! of 
those who did not. At the other end of the scale, 2%% of those 

Reporting opposing viewpoints became heavy users', vs. 105 of those 

\ f 
rfho s^id not. However, once again we are t wprking wJLth small numbers 

^1 reporting this form of competition;, and even though there is a 

moderate correlation (gamma* .36), it is not . statistically 

^ significant Cp 3 .^o;. 

There *ere no significant relationships *ith any of the jother reasons 
for* con£eti£ion Included In the oheckllst. 
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TABLE £-10 

PERCEIVED UNETHICAL COMPETITIVE BEHAVIOR VS. SUBSEQUENT USE Q? EIES 
? t . ■ 

/ * 



Houjrs* Used 


Yes 


No 

1 


• ■ 


715 


265 


m 

5-19 




305 


20-49 




295 


50+ 


0 


155 


Number 


- 7 


66 




Shi 


square 3 6.5, p 3 .16 






gamma 3 . 6ii 



Question: How 'would you rate the degree or Intensity of competition 
within yaur research specialty? . . . v What are the reasons for this 
ocmpetitiori '(check, all that apply). 

Scarcity af or competition for funds 

RIv&l groups of collaborators # * 

I 

High achievement or success drive of persons in the field 

Some persohs act unethically m 1 ^ 

Strongly opposing views „ 
■ • 

x -Other ♦ * * *. j- 



* .. . • 

•- • WE, 



' TABIE, 2-11 

WEAX OR INSIGNIFICANT CORRELATIONS WITH. tfO(jRS"OF USE 



1 Question 

s * * * • . 
Houfrs/week .in* specialty 

.dumber of co-auth"or\ \n last year 

jixtent to which scientist considers 

self ir> f imartist ream" 

Total of contacts In specialty 

Frequency of anticipation 

.Concern -£bout anticipation * * 

Extent to which emotional commitment 

- "."governs owh beh^tjlor 

•Extent to whlcn etadt^orjal commitment * 
'"•ought to gbvern^behavio^ \ 
V m Extent to which* irrelevancy of « 
* _oersonal«attributes governs 'own ■ ■ 

- \ behavior 

Extent to ;whi«:rf Irrelevancy of ' t " v 
personal tet tributes ought tci 'govern 
' *■ behavior t" . - * * • , " 



' N $amma 
.03 
■ 

- . 12 



13 

06 

19' 

24 



Education. • 11 % ■ 

- tj* • * 

Yeatvs since highest degree * 
3qok3' ctyrjjentiy in progress ^ 
Total a^t^'cles published .during career 
Papers presented \&at'yea t v\ , < 
Total productivity scale . ' 

> • . v. 

• - • • . ■ 

Pref erenc'e- f or working in established' 
. - . - . \ 

areas" v - . 

How well known in field 



t 



-.09 



Eta. 
•?7 



h.31- 



i .20 



.47 . 
.20 

.11 



.23 
.80 

.'53 

. 14 ' 
.04 

--02 



.22 
.26 
:19 



Whether EISS will affect familiarity 



.16- . 

fii 

* : ' 

.31 " 



.21 



'* »*33 ; 

.29 
,10 

';71 
.06 
• 5-2 
.49 

i • 

' .62 * 
.68 



0 



. Hitjh on^is work. 



. • X " * • 

i Reading speed^ A 
Typing speed A * # 

Preference for writing vs«, speaking 
# Somputers are (wcfhderf ul/terri&le); 
N^^ Previous use of message. system 

Previous use of terminals to play games 
Access to terminal * 



Trust computers 
Perceiyed pressure to use the 




stem 



Anticipated usefulness of group . * 
conflrences , * 

^Anticipated usefulness of %ext editing* 



1 0 


• oo 


♦ - ■ 

% 17 






• 

♦ 71 


• 

.07 




• 11 


• 13 


. 14 


A it ~ 


♦ 


• 39 >/ 


.Q6 


'.93 


* 


.1*4 

r 


.16 




:2o 


« 

.01- 



Source: ?re-Use questionnaire (See appendix *fo^ wording) 
ilotes: Gamma notes lir^ar relationship 
Sta denotes curvilinear * relationship * 
v "p" is significance level, determined by % Chi square t;est 
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Compliance oregsure' 



At pre-use, u^ep^ier^^a^ked to indicate whether they were /'required" 
to use the system touly three checked this response, had been, 
requested to do' so (a form of pressure), or were 'free to use it as 
little or as^much ^s, they chose* Pressure to use*a system like ) this 

• * a 

seems to have the reveree effect* Among' those who felt that they had 
been requested toVse the system, there rfere more, dropouts than among « 
thpse who* perceived * free 'choice* and thfere were no heavy users in 
this "non-f ree'x^joice" group* , i w 



\ 



Failed Predictors of U§e 



A number ' of other variables wpre hypothesized to aff ept amount; of use 
opoji computerised conf erenclrj^ system. The hypotheses were tested" by 
iifcfcluding indicafons of* the£e variables the pre-*use questionnaire, 
^and cross-tabulating them by' number of «hours orr liae/at the ^ime of 

- V ■ 

the \ follow up questionnaire. The following variable^ * ar^ nop 
significantly related t^o~ time online (as measured by division into 

the* 1 categories, 1-ess 1 trtan five hours, 5 L 1Q hours, 20-49/ 50-99, and 

• .V- •» • 

100+) : - . . ' ■ 

m « „ • 



1. Hoars per w,eek spent working on the specialty; or any other' 
reported* use of timp gathered in the pre-use questionnaire . ^ *. , 



• c ^ * 

2. Frequency of previous -anticipation or concern about future 

anticipation 6f one's work by others /who publish similar things 
1 • t * 



J Age ' (There are too fev/ users under thirty or over 50 to. adequately 



pest this relationship.)* 




4. productivity in terms of reported books, articles, etc., either in 



the 
with 



previous *«y ear or in onp 1 ^ total career/ Although correlations 
productivity Aeasures* are no« statistically significant, *ther»e 



t al*e 3ome moderate correlations. , 



They; tend t& suggest curvilinear 



p att e r n . m o r e^ — than .a lineal one. > That is^ 

publication levfeis before use' of EIES tend to 



;e — w^th -modehatk!. 



use k the ^system more, 

~This makes some sense', thbse already publishing ver^ h(eacvil^ probably. 



do not need* any new information resources or professional contacts* 



i. Pf^e 



ef erence f.or wording in established areas of science. 

6. Subjective report of how well * Known the Aw^oer is in his/her 
specialty., Th^s is, contrary to "tne hypothesis thaj those vho are 
"low" in the status hierarchy will be more strongly motivated to ase 
toe system. However, as will*be discussed below, a group aggregation 
of this variable does fcave^spme predictive p.ower — A scientific group 
seems to p.eed a certain number \ of "starsT" 'to niotivkt^all of its 
participants. 

7. Whetner the* thought this use of 2IES would afTect how, well known 
tHey are in their research specialty ™ * , / 



-8.vReading speed , 



9. 'Speaking vs. writing skill* The question here Vas whether the 
prospective user ' thought/ thfit he t>r stte was more effective when 

* writing or speaking- Almost exactly the same proportions of the two 
f types (speakers vs. writers, as self-assesse^ before system use) 
became dropouts Qr heavy users. ' v 

10. Typing speed 




11 . -Atti-vud^s—t QHA r ds co m pa t ena { e i ther o n, a "■rfnn djsri'uLJifl — SLarrJL WLeJ' 
st>aJ.e,' or in terms of trusting then to hold the daily working fiLes 

that one needs), i - ♦ * 4 

i \ . * * 

v • ■ ' / ♦ 



12. Previous use of computers or terminals. Neither any of the 
individual items, nor a combined'* index on^ total previous use* of 
terminals was*a* significant predictor,. - i 

The. correlations and significance levels for .these and o'ther "failed 
predictors* are shown in Table ,2-11. There Is a suggestion that, 
those who placed a high value on the unique features of ZIES^as 
compared to a message system v group* *conf er.ences and text editing 
features; are likely to use the -system -uore. This LS similar \to t^e 
finding tnat expectations about the system's overall usefulness 
nelped to predict hours on line- : M ■ 

L . 

There is weak support for a relationship between basis values and 

f 

subsequent use. The ore-use questionnaire contained sets of 

* ♦ 

questions on two of the "pattern vari^sles 11 used by Talcott Parsons 

and many subsequent sociologists to characterize value patterns* 

» 

These ane "universalism 1 * vs. "particularism* 1 (whether a scientist or 
his/ner work is judged* scft^ly on the basis of their work,, or 3olely 
on the basis of wh'o they are, jm terns of personal " knowledge, of or 
relationships [with the person) and "a/fectivity-aff eative. neutrality" 
t Whether. J Scientist is ew^tionally committed to his/her' theories , or 

* r X? 

totally objective and not emotionally involved Kith nis/her 

- 1 ' ' 4 7 

scientific theories.) , . «— 

/ \ * v. 

^ ■ 

There are w^a*c relationships' showing , some tendency for tho^e placing 

their answers at the ^emotional commitment-" end of .the sdales to use 

■ • . * V 
EIES • more; and for those in the "balanced" area of the choice between 

the relevancy and irrelevancy odf personal attributes for Judging 

•v • ' ' — 

scientific work to use it more than those at either extrerte- These 



results are suggestive of possible relationships* but not strong or 
consistent enough to say that rfe h^ve proven that such a relationship 
does' ejcist, * 

Collective Group Status 

« i 

Altnougft there is no relationship between the self-assessed status o€ 
tne indi/idual t unknown to top« of field;, there does seen to be a 
relationship *with trie collective * status of the group* As shown 
oelow, ' the .groups that had tne largest^ proportion of well ^known 
fibers tended <on ttie, average to have tne heaviest aser? of :iie 

tny OTHER £H0U? 
relatively high 



system. 


What matters to 


tne individua 


1 Is 


how 


MEMBERS 


availaole to , 


communicate 


with 


have 


professional status. 


* 


V 




3rou*p 


5 jii 3 rank 


, 3 Hi Use 


rank 




30 - 


HZZ • 1 




' 1 




40 


* -245 • 2 


345 . 


f ' 3 




45 


' 235 3 


175 






35 

• 


22? 4 


435 






54 

* 


' 145 5 


105 

* 


5 





Note: w HiS lf standi forrf^^Tve^^ of group members ranking 

themselves as 6 or 7 on OJie- seven-point professional ^status scale* 

' . ' - a' 

n Hi use" is the propor tiohsjof group members* using 50 or moreihours of 
line' time by theTollow up questionnaire. 



7* < • < 

Pre- Use Terminal Access ' . ^ • 

, ■ ' V 

Although terminal access was reported as an important barrier to use 

by about 20% of subjects, £here was no overall statistically 

significant association between*- the terminal aocess situation 

* reported at pre-use and amount of' time spent on line by the fi?st 

folloW-up. Many participants were given use of a portable SIES^ 

■ ■ 

terminal; m tnese were all persons who had reported no access to a 

■ * 

ccQDuter terminal unless one were provided for them* This meant that 

* ** 

tney hati a light-weignt, 30 cps printing terminal availably both for 
office and f-or home use." 

^hat rfe find «are some puzzling negative relationsips rfltTx^ter**Inal 

f — actress and characteristics* For example *e find the-f^l lowing: 

Home access " % dropouts 

Reoort terminal at /none" * ft 035 

t 

Report terminal available to f 323 - \ 



;ake nome 



or nom 



x Io terminal aval laol^r or nome 2b 



, There was likewise no relationsnip witn printing speed gt # the 
terminal, though one would be expected. Another puzzling relations!? 
is that the highest proportion of dropouts occurred among those 
reporting * access to both a CRT and a hard copy, terminal; ratner than 
only dfce. This seems an ideal terminal arrangemAt for ifse of EI2S. 
j One * possible explanation is thatvj^hose in a termin^l-ric^axi^i^ 

are • also in an already computer-resources rich* environment, and do 
not need additional resources such as EIES." fc * ♦ 

S , d 

f _ M 

i ' a ■ 

* It is .certainly not likely that having a terminal at,, home or two 



f 



.terminals in the bffiee caused less use of "the Systey, but" rather 
that motivational factors are simply much more important for the 
scientists in this study. For example, one*member apologized for not 
using the system more because he had to. drive about an hour each way 
to use a terminal--- and he was logging over ten ft£>urs & mdnth! We 

^ave a curio^ disjunction between the lac* of relationship .between 
tne terminal access situation at pre-use not being related to level 
of ase of tne system, and. a fairly important role for. subjectively 
reported terminal "Recess oarciers at follow-up. What prooaoly, 
happened is that strongly motivated users witn ooor access expendea 
tne time or money to improve their terminal a9cess situation. But 
g^od , terminal ^ ac<j.es^L_alone, . witnout JEOtivatJ^c^ will, not lead' to use 

.of tne system. In otner words, tnere is- an 'interaction between 
terminal access and motivational factors.* " Thus, 1 the overall 
conclusion r^acned about the importance of terminal eccess to system 
-cse, given the findings on tne followup questionnaire as w*^l- as tne 

aobya observations, is that the relationship is conditional on 

» ■ • 

motivational factors.. If motivation is wea*c, doop access becomes a 

barrier that may be decisive in limiting*use of the system, on the 
>other hand, if there is no access .at all, gven high aotiVa-ti'on cannot 
lead to higiv system use. , , • 
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T£BLE 2-12 t , / 

• Whether Individual Teacher or Written Material only t 

Were Used,tn Learning cJES, by 

• ■ 

, Accumulate^ Time on Line at^Fol^ow Up * / * 



'Hours *Used 
% <20 hours* ■ 

50-99 
Total 

M responding 



Live Teacher ' Written Only, 



563 

31 

■6 

* 

1003 
32 



J- 



3^ 

i. 

14 

3 

)0% 
77 



Source: Follow-up questionnaire and monitor; data 
Ch^SquaVe* 22.4, o p*.03 

Question: Did someone demonstrate *EIE5 to^you in person, or did you, 
> teahn from' the. written materials? * * 

% . " • 

Live ^eacher " 

Written material only '« 



0 



9 
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77 



» 

91 



The Effectiveness' of a Human Teacher 



It rfas hypothesized .that thq^e who had some personal training from 

another person frould be more likely to learn" Jthe system and become 

regular users. There are many .reasons for this. g One is that 

personal .training should be more enjoyable. The second is that it can 

be" tailored to* the questions' and difficulties of the individual./ 

\ 

• 4 

There is every - reason to believe t-nat the personal teacher should be 
superior to simply receiving £ large*, standard document in the mail 



and teaching oneself, 

* • 4 

However, the dat>a in Tab^e 2-12 do not,support phis. In fact,' there 
is a statistically significant dlffer^rtSe i?l Tne^^Q^er direction — 
those wh6 rial only r, Kow to. Use 5IZS 1T and the use of on-line user 
* consultants were less lilceiy to^eiome dropouts fif low users - than 
those who had some oersonal instruction* \ t 

• • • 1 • • . 

We do not ipcept this as cause^and effect. For ofce thj.ng, there are 
no data about tne extent and quality of the personal training # that 
was received- Secondly, it, ftay be that *t;hose users who were the most I 
confused and negative were the most likely to seel^va personal 

* 4 ' \ 

* training session, anc^ that Without .Such p'ersonal attention from art 
experience^ user, they would, have been even more likely not to accept 

<' ' 

Personal training Xs expensive 'and -time consuming* The eviderjfre JTr<jm 
•this, study does not Justify such ^x*penditures. However,- a controlled f 
. experiment with random assignment or subjects to different Kinds of 



teaching materials ( x live teacher,' rfritVen documentation, on-line 
lessons of an interaotive nature) rfouid be necessary in order to' 
estaolisn the relative effectiveness of these training methods for 
different types of users. ThV on-line lesson day well be the most 
effective method of all, judging from the many spontaneous requests 
received from users for this sort of aid, and from the fact that in 
controlled experiments, first-time "users were able tc? learn to enter 
> and receive material from EIES in about 20 minutes, with an 
interactive lesson on line. ^ * . * 

COMPARATIVE RESULTS ?0R' A STUDY- 0? flLS 'ft ; / 

* 

3wen Edwards reports extensive data ^on the correlates of amount of 
ase of N'LS, a computer-based text prncje^Lalflg_ and communications 

system, Me rfill examine the results in some detail because it^is the 

* * 

only other puolicly available study rfnich examines a wide range of. 
%* variables ins relation to .acceptance of a computer-based communication 
system. * P ' \ ' 



^13 i? a general office support systea 



1 • / - ' 

. •Particularly when used in 



conjunction wjLtii an intelligent terminal rfith a special "mousb" 
.device for pointing during 'editing, it is excellent for document 




or ocu 



ex 

TV 4 



Action. It also includes thrse communications capabilities, to. 
hang&. r mSssage^asynchronously , in real time, 'or to exchange fil$s. 
Joes- not include /a conferencing* component or other structures 



.meant to maximize group communication and% exchange. 



1 



Edwards 1 * X 1377 ) study was'*based on a questionnaire sent to 2$Q users 
of HLS in thirteen organizations. Ninety four, or 3tfS, responded. 



.Of these, 3Ww were managers, 42£ researchers, and 2ti% support staff. 
Some of tne researchers also had a supervisory role, as a total of 3 
405 reported some supervisory responsibility . 



The NLS setting was quite different from the funct/on for* which EIES 

was used during" the^ operat ional. trials. It wsfs used as a tod to 

directly support the ^regular, paid job. / It is therefore most 

important in increasing the gerYeralizabil'ity of the EIES findings 

* * v 
tnat *aany of Edwards' findings about the importance ^ of <aptitudinal 

/ 

variables are, similar. A copy *of ^iwar;3s T questionnaire was made 

■* 

available during the^ design phase of this study. Many of the items 

were borrowed to increase tiiS direct comparaoiiity of the findings of 

tne two studies. For example, the scale of useless/revolutionary was 

Edwards'. Though results for attitudiftal variables measured" with the 

Sa£e" question are similar, there are some contradictory findings for 

other variables. 1 fhe explanation may be that ^the* specific questions 

used were .quite different; or, the differences may be attributable to 

use by an office staff to support their work on the job vs. use by 

* 

academics 1 to support their informal, organizationally external 



communication.' Still a third source of possible differences in 
findings are diTFeCiences between the systems. NLS was a fairly 

V ■* 

\ t% * • 

\cc*nplex, command driven system for m augmentation of the .Indi/idual 

♦ ^nowl^dse worker/' whicn included some ' corrynuni cat fotis .components. 
EIES is primarily a communications system jflth some text editing, and 
with a simple menu-driven interface for beginners, pho have no, desire 
to master the full power available on the system/ 

■\ 

* * 4 

*£dwards f "report frequently- gives results-£or parts, of the sample, as 



ell as the wSiole sample. Results are reported for both total or 
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"general" use, •and for Just communications use* Sometimes results 
are reported separately for - supervisory and non-supervisory 
personnel, Since this was found to be an important variable aff-ec^ing 
us? and .attitudes. In looking at correlates of usage, the * dependent 
variable "GENERAL USAGE" -was broker^ into three ordinally ranked 
classes: "Low 1 " usage of less than one hour a day (28?); "Medium" 

* usage 4 of one to three hours a* day ^31*), and "high"' usage of three or 

* ■ * 

aqce nours per day 141JJ). Notts that the* "middle" level usage of MLS , 
for this study would constitute "high" usage on EIES; - ^ 



Since Edwards' sXudy was a Single cross secti-tfa, it is difficult to- 
Identify .cause and effect. For. example, when, she reports" that 
perceptions *of increased productivity are associated witn more Use, 
we 3o not Know if there was aji expectation of ris^creased productivity 
before use, the growth of this perception as \ result of use, or a 
combination of ooth. Some pf^Eawards ' 'findings are omitted from this 
summary because fchey seemed to deal more with perceived impacts as a 
result of use than with, attitudinal aauses of use* 

Edwards reports that general attitudinal and access variables are 

most highly related to amount of use of NLS. The x strongest 

correlation X^amirfc* *69) overall was between jus e oX a terminal at 

home and amount of use. Typing skill wa§' found to be related to use" 

of NLS' othj -among those who had negative perception of [ the system 

(g^mma ».68j). Am^^ those with medium)* to highly posi^t^ve perceptions 

of the system,- thef*e was no relationship between typing skill and 

amount of use (gamma* #05) • Edwards states that "Once the perceptual 

f • ■/ * * 

barrier is crossed, typing skill is irrelevant Usage. " , She also 
suggests that "we can recommend that when implementing an Office - pf 



i 



r — 



Jill be 



the Futur'e system, it will be beneflc.lal to convince potential users 
.that they *need rtot know how -to 4>ype to make effective use of tl?e 
system" 4^). 



The other variables wnlch are rqost strongly related to - total use are 
tnos^wh-Ich indicate perceptions of utility of NLS: - - 

1. "Professional Image": There is a gamma of- .50 -between the 
perception that; use. of Itts wiil^lmprove one's professional Image* and 
amount use. This Is a Variable j which was-not found to oe a 
predictor for t>he scientists on^EIES. impossible explanation Is that 
tne opinion ft of~ one's organizational, peers is nucn more- Important to 
one's future career than the opinions -of scientific oeers on other 
academic campuses, f who, after all, ,do net sit on- one's tenure, or 



'promotion decision -making group. 



2. Perfeeived Impact on productivity: gamma - .49. This is irrea-sured 
with an identical question In the" EXES study. The correlations are 
sLuilar In direction but stronger for 11LS. ^ , * 



3*„ It is related positively to the perceptlph that NLS use increases 
the accessibility .and visibility of one's work to others (gamma= .44)' 

4. Ther^e is a moderate relationship with the aser',s initial 
perception of the system anct subsequent general use (.35) • There Is 



also a moderate relations"/ with 



/ * .terminal.' / 
A 



training", and sophistication of the 



Generally, correlation^ with communications use are simila> to but " 

ERIC 
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. N "weaker "than those 'with general or total .use. However, me ; 
* * ♦ • * * 

Interesting' -exception is sharing a terminal.- ■ It. does n6t affect 
general .use, . but having ' a shared terminal does impact'; on 
,, communications • use ' negatively. . *• Another .difference is*, privacy: 
' - concern over- It influences communications use*much more than/ general 
use. ~ . v ' 



The correlations for training and. terminal sophistication probably* 
can be explained by the'' greater complexity of the NLS system for 

•beginners. , At the time of the study, it was- j command- driven, and 
designed to be used on a/sophisticated terminal rather than a simple . 
one,. ( It is not likely that a beginner, could^ learn NLS with^ no 
training or personal contact; whatsoever with an experienced user. Ori 
the other hand, EIES was designed" for use on a simple terminal, and % 

-to be' usable, by a beginner in a menu:" drivefT mode without any formal 
training or personal instruction. In other words, the differences fop, 
these variables may be attributable to system differences. 
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Table 2 r U * ' • * ' 
.* i • VARIABLES .USEP IN SDWARtiS' 1 "NLS- STUDY 

ACCESS*- user % indicates that thfere*was or was not difficulty accessing 
the system K ; - 

ACCESSIBILITY ' OP WORK- on a Tive#oint Likert. scale, > thV degree to 
which the accessibility" of the user ? s work to others is perceived to 
have ir^creased^or/decreased 

COMMUNICATIONS J USAGE- 'frequeney .of us£. of the system- - for 
communications purposes (exchange of messages, doc\iments, linking in 
.real* time) . - ' . • : 

DIRECT/INDIRECT^ USAGE- direct interaction on the terminal vs. using 
the system *via support staff , * 

GENERAL USAGE- Total hours. $3* week ♦ ^ . V 

GROUP INCENTIVl£ -use 'is required; requested, or the user feels free 
to use the system as he or she chooses 

£pME USAGE- individual 'does or does 'not occasionally use ^>-ierminal 
^rora home . * , ' . 

IMAGE- on a five-point scale, the degree to which Jthe' user believes' 
hi* or her -professional image has been increased or decreased 

INITIAL PERCEPTION- the user's retrospective- react/ion to the system 
when it wagvfirst introduced (thought it *ould be useless, thdught it 
wo^ild revolutionize work/communicati'on processes) 

INVOLVEMENT?- the. user *as or was not involved Tn\the decision to 
subscribe to NLS " , • # ^ 

PERCERTION-^ftrf index 6 oris trusted from questions on current perception 
.9_£jfch# usefulness of NLS -(same *$ls initial .perception scale, above)*; 
and five-point attitude scale? on { compat ibility-4hcbn>patibilitjr of 
the system to normal .WorkingAwri ting/ thinking Organizing Style} 
flexibility" vs. - inflexibility , of - the \ system; 

reliability-unreliability of tbs> system • • 

— v ■ W ' " * 

POSITION- support staff, research, management , 

PRIVACY- individual doesn't use the sy£tfcm for work of a confidential 
pature; takes precautions' to ensure the conf identiallt^sof work, such 
as changing password; or does not let the privacy aspect affect u^e 

PRODUCTIVITY- A five-point scale, the degree to which 'a user * believed 
his or £er work' efficiency/productivity 'decreased or 'increased afc^ a 
result of using the system • ' 

PROFESSIONAL IMAGE- believe' that the system increased * or decreased 
professional image '» , 



PROXIMITY-, the distance between, the closest available terminal and 
the. user's 'office, defined as in tfie ofifice, within 50'feet, or mope 
\ than fifty feet from, the user?s place of work - , , ' 

• • * > 

QUALITyV . A. t five-point 3cale\ the degree'to. which a user believes Ithe 

quality of'' -his or/her work *has| Increased or_ decreased .as a result of 

using the system ; " . ;' ' , 

•**'».•' *.*'''• 
. SHARING- the individual has sdle' or 'shared .use of the terminal 

' * - ' : '' - / ■ ' r - / I- , • 

SUPERVISION- the user does or c^oes not supervise other employees 

TELECONFERENCE- t^e user -has or ^ haS not ever, participated ''in a 
teleconference • c . * / , K 

TERMINAL TYPE-, teletype only 4 CRT with -teletype version; display 
based versidn of NLS with special terminal^ and electronic 6'ursor 

TRAINING^ formal program, grained by other Employee » in* charge of 
training; by\other u^ers of MLS; or. no* training ^program 

TYPING ' SKIlfL- the individual does -.or does not claim to know how to 
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T«ble^2-14 ' y 

. ■ ■ ■ " • "■ ■ y ... 

Correlations (gamma> with Generalise and Communications Use of NLS 



Variable 

it' 


Genusage 


Comusa'ge 


., POSITION 


-.10 


.08 

# 


SUPERVISION - ' ■ \ 


-.21 


r.30 


v INVOLVEMENT * 


-.37 


-.22 


GROU» INCENTIVE 


' -.05 


* 


• TRAINING 




.23 


TYPING . > * 


-.3« . 


'.-2*2 


TELECONFERENCES 


-.22 


-.50. 

* 


TERMINAL PROXIMITY 


.05 


-.23 


TERMINAL TYPE 


. . 4 1 • 


..48 


. -SHARING 


14 , 


-.40 , 


DIRECT-INDIRECT USAGE 


.18 ■ 


-.01 


ACCESS -PROBLEMS 


-.18 


-.01 


PRIVACY 

• 




'■ --.43 




A .35 


' . 27 


■ PERCEPTION INDEX 


y -?* 


.24 . 


PROCESSIONAL IMAGE 


• i .50". 


..49- 


ACCESSIBILITY^ 


/ % . .44 


•35 


PRODUCTIVITY 


7 1 
/ .49 


• • .38 


QUALITY 


I - 3B * 


.12 


HOME USAGE ' 


1 . -.69 


-.52 



Source: Edwards, 'An Analysis of Useage and Related Perceptions of NLS,- 
p. 43 
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TABLE 2-15 
SUMMARY 0* FINDINGS \P0R EIES AND NLS 



iMDirain 



tACTERISTlCS 



Variable 



EIES 



* i 



NLS* 



. A» Attltudinal Variables 
Attitudes ^toward task 
a) relative "importance or priority 

Attitudes toward media 
a) attitudes towards computers in 
general • j 

J>) expectations about the specific sy 
* 1) Anticipated amount of use 
2) anticipated impacts on 
productivity I 
Expectations a,bout how system use 
will affect relationships with the group 
• Perceived 1 pressure * to use the system 



strong 
none 

stem 

strong 
moderate 

weak 

weak 

( negative) • 




•B. Work Related Skills -and Characteristics 
1. Personal communication skills 

a) reading speed * 

b) typing .speed , . . 

6) preference for, speaking or writing v 
.2. Previous feinted experience 

a) experience using cqmputers 

b) use of. computer terminals 

c) v use of other computer based 
communication systems 

3. Productivity • \ 
a) Hours per week worked* 
b) Nitober of, publications or other ■ - 
output measures, \ J 



none 

none conditional 
none' 



C. Connectivity 
Number' of persons in field 
Vhom one is in contact " * 

* . NumbeV of' persons on system with 
whom/ one was in previous contact 
> TNo one" to communicate with 
/ How well known person \$ in field 
;/ Whether >a scientist feels' "in the* 
♦ mainstream" or not 

Number ot coauthors (or coworkers) 



non/ • 

none 

none 



none % 
weak 



' *D. Bemographic cWaracteris ti 
Age \ \ 




I 

I weak 

moderate- j 
' strong, 
moderate 
- weak 
weak 

weak ' 

( 

u 

rvilinear) 



none 



moderate 

* 

X 
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s Sex 

» Educational levei ' : \ 

E. Environmental variables 
'Position in the organization (or* 
status -in informal group) 

Amount of pressure to use the^ system 
(from superiors and peers) 

P.- Basic , values (? t g. the pattern 
va'riabless: universalism vs. 
particularism, affectivity vs. 
affective neutrality) 



none 
none 



none 



weak- 
negative. 

weak 



none t 



SYSTEM CHARACTERISTICS 



VARIABLE 



In-{lerson or forrnal training, vs. 
documentation only 

Quality of the Telenet interface 

Whether or not the system was a source 
of difficulties 

System availability ('downtime during 
workday or unavailability nights and 
weekends) % 

Trouble with the telephone or high 
cost of long distance telephone due to 
absence of Telenet node 

1* Access to terminal (subjective) 

2. Pre-use access to terminals 

a. Own or share at office 

b. Terminal for use at home 

c. CRT, print,' or both 

d. Sizet and weight and printing speed 
of the terminal(s) available 



EIES 



none 

/ 



moderate 
moderate 

moderate 



rpQderate 



moderate 

none 
' none 
none 
none 



GROUP CHARACTERISTICS 



STRUCTURE < 
Size 
COHESIVENESS 
Competitiveness * - 

Trust, or openness amopg members 
Status (are most group members 
prestigious in their fields, or not?) 



noffe : 



none to 

. weak 
strong 
.moderate 



/ 



NLS 



moderate 



none 



moderate ■ 
strong 
moderate 



/ 



o 
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< SmiMftRX/ AND CONCLUSIONS ' ' . 

. , ..?r \ • 

The results f^or variables observed- in 'this study and the NLS study 
are summarized in Ta ble 2-15 • 



Motivational variables are most >strorjgly associated with level of use^ 

of the EIES system, r,ather than characteristics of the system itself. 

The most important reason gi-ven by users to explain limited use of 

EIES t is that other, off-line professional activities must take higher 

priority. h The relative, priority v of ElSS-related and - other 

professional, work was by far tfh'e most important reason given both in 

'the checklist on the follow-up questionnaire and in the post-use 

» 

open-ended question. * 

_ j$ * 

• ■ 

The strongest .observed correlate of the level of use is the • 
ANTICIPATED level of 'use before experiencing the 'System 'at all. This' 
variable is a conglomerate of , individual attitudes and expectations, 
probably including relative importance /to, the person of cbmmuni eating 
with others in the EIES group and. amount of time available for such 
activities after the more mandatory Job-related tasks are completed. 

I 

, - _ 

Measures of. connectivity (pre-existing communication ties with other 
group participants) also appear important. An item on the pre-use 
questionnaire. (number of group members previously known) yielded the 
highest Pearsoa's correlation coefficient with total hours of use at 
follotf-fup. An item on the followup self-reporting checklist ("There 
is no one. on th^s system with whom I wish to communicate a greats 
deal") yielded the highest correlation coefficient with level of use 
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of any of the self-reported reasons, 



Access barriers .as a class- (including, access to a terminal, trouble 
wifely Telenet and system 'uli^va^Labillty ) are the second ranking/ ty£e 
*of factor related to amount of use of E£ES« However, it must be 
noted that with the exception of terminal access, the perception t>f 
other access barriers is more an effect of moderate to high. use than 
a cauie of drop-out or low use behavior: the higher the* level pf use, 
/th£ rrfere frequently these barriers were ' indicated to be "very 
important". 



Among the variables which were hypothesised N ^o/be positively related 

to'level * of use, but which are not significantly related,^ are receipt 

of personal* training, reading and typing speed, attitudes toward, 

computers, previous experience with computer terminals or message 

' systems, 'and how well known the person was in the specialty. On the « 

other hand, groups that'' were composed of a higtf proportion of 

high-status" members were, on the! average, 1 more active than ^groups 

which had a small proportion of well known members. 
* * 

i 

* 

In 'comparing the findings to a similar .study of determinants of 
r amount of use' of' NLS', we found that attitudes and perceptions were . 
, important predictors for both systems and types of ushers/ and that 
typing skills are .not a' prerequisite for high levels of use.-. 
.Terminal aodess and special training were more important for NLS. Qn 
the other hand, access barriers such as telephone or packet switching 
network (Telenet) problems ,and system unavailability nighty and 
weekends during the. first year were moderately important barriers to 
• uSe of EIES, but not included in the NLS study. 
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The Importance of pre-use motivational and perceptual factors 
suggests tlpat there may 'be some" important underlying psychological^ 
traits that may predict, acceptance of systems^such as. EIES. An 
indirect indicator of this is- Jthp finding- that scientists- at the 
fiddle -'levels of productivity* and connectivity within the specialty 
tend to use .the system <more*. This may reflect « achievement 
orientation /or striving on their part to improve their professional 
standing. This observation has* led § to 'plans 4 to put some basic 
psychological tests pn line on EISS in the future, administer them to 
new users, and see to* what extent they predict use of the system 
during the first thre^ to six months, , 

Multi-Variate Analysis: Stepwise Multiple Regression < ' 

Multiple regVession .is a general statistical, technique -that cam allow 
us to' analyze the relative-, importance of =s the various strong 
predictors that ■ we "have identified, and ' to "describe their 
int eractidffL , \ 

j\ stepwise multiple regression was chosen as the best technique for 

» 

examining interactions among the identified causal factory. A 
forward stepwise inclusion technique was employed. The # orOer -of 
inclusion is determined by the* "respective contribution of each 
variable to explained* variance in the dependent variable. The first 
variable entered is the one. that singly explains the great e*t .amount 

of variance: the variable that explains the greatest amount . of 

♦ «✓ 

variance in conjunction with the first is entered second, and so 
forth, until no improvements can be made in the prediction. A "Another. 

' , 91 ' Wo . • V • . 



J r 

way to describe, mt happens is that the 'variable chosen at each step 
is the o^ie whicfr explains the greatest . amount of variance* which is 
still ^unexplained by the variables already entered into the equation 
at previous steps. ' , 

. - • 

The independent variables chosen fpr inclusion sere defined at the top 
of table 2-16. Two .separate analyses are presented: the-sfirst for 
prediction ojft LEVEL of use -(with 50+ hours as the top category^; . and 
th^Tsecond* ' f or absolute number of hours of use. The latter analysis 
will favor, variables which help to explain those with very high hours 
of use. A third analysis used the log of the number of hours; its 
results wer^Syery similar to that for leVel of use. 



The advantage of this technique is that it, allows us to compare the ^ 
strength of the preuse predictors with that of the, self-reported 
reasons, and to examine interactions among facers that atay . 
themselves be highly interrelated. __.The_ . disadyantage.i^ls tha.t-.the 
number of cases is greatly reauced*; only those who answered all 
questions on the pre-use and follow-up questionnaires are available^ 

for Inclusion in the analysis. This reduces our data base to only 65 

! " * ' 

cases. : ' . 

> 

We can see from the correlation matrix in Table 2-16 that the best 

overall predictor of level of use is the estimated number of hours of 

■ » 

use per week, made before using the system. In turn,*the highest 
correlate *with«,this estimate -is the number of other group menBers who 
were already known, ^ Before . signing on. This gives us some insight' 
into one of the probable strong determinants 'of this ( initial 
estimaJ^^- the expectation that the system could . be used to increase 



communication with * colleagues with whom onjs had. valued ties. 



>Qne Of the characteristics of the .stepwise procedure is that two 
highly correlated predictors are* probably explaining '[the same" 

variance; therefore',' if one is chosen at. step one/ Jt not lik'ely 

r - I 

. that t?he second can make a great deal more contribution^ * * 

. ' >) - , " ' " * . 1 ' • 

We also ncfrte that " al thought ,f other .professional activities" is the 

most frequently* off ered explanation for limited use of iEIES/ it in 

* \ " 1 \ 

£act has little relationship touevel Of use. * 5' 

" * . \ % 

Turning to the results^ of* the regression, we can look af.-the order of 

factors, # the extent to which the \inclusion of *.e$ch increases the 

~ multiple regression coefficient (MULT R) gtnd its square (R SQUARE) , 

which is the proportion of total variance in level of^use that has 

been explained by the /variables, included 'in the equation at each 

step-. BETA is £he standardized regression coefficient. 



After initial estimates of use, the variable whioh accounts for the 
most variance in_ level of- use is, the "NO ONE" to communicate with 
factor. One might expect this to be highly '(negatively) related to 
.the number of persons knoftn before use; the fact that it is not 
suggests* that there was a divergence for many between the expectation 
of who would be available on* "line, anc^wtio actually was there to 
communicate with. This of course fits in Well with the ^otraerved high 
"drop* out" , rate* In'other words, the prospective user knew who was 
. Expected to be available on line and used this to estimate amount of 
use of ttje system; however' many of thfe anticipated^ communication 
partners Were among the "drop outs*" <leavihg many group members with 



1 \ K 



the feeling that* th$re* was "no one" left with whom they wished to 1 

communicate. . . . ^ 

* • 

v 'This second predictor (NO ONE) raises the proportion of variance 
explained fronf 21% to 26S.' None of the othfir variables make, much of 
^ an improvement ? in quf ability * to* predJ^J^. * For instance, though 
jDeroeived- problems with ./terminal access • is* selected- as 'the be$t ^ 
predictror to' be' added on th$, third step, it, only increases . explained 



variance 'by .IS. * 



For analysis of absolute number of hours of usfc, rather than level of 

i * * 

, use, the number of group members known before the beginning of the 
computerized conferencing activity is the best predictor." It 
explains 23S of the variance. We can* deduce from the difference 
between this - and the previous analysis that those who A knew many other 
group members before using EIES are likely to use k very high numbed 
of liours on line, communicating with all of these colleagues. 

■ Subsequent steps of the analysis are very similar -to those <£or * lavel . 

of use: estimated * hours improves th£ prediction significantly, 

' J - 

followed «by small .improvements added by tjie terminal •afccess and "no 

» » 
one™ variables/ and the B .subjectively reported "other activities" 

makes no objective difference at a]A.; Altogether, the four variables 

w / * 

entered into the equation yield a multiple correlation coefficient of <% 
.62, corresponding to 3&S 4 0f *the/ observed total variance'in hours on 
J o l<Lne. , ^ There is .thus still considerable ^"unejcplained" * variance in 
hours A 0f ase-/ not accounted- for /by the variables included in this 
study. ' \ • 
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, Table. 2-16 ■ 

* • . * * * 

Stepwise/tultiple Regression, 
eterminants, of Level of Use 



:ori5 



LEVEL*Olumber of "hours on line at folltfw up, categorized as"*<5, 5-19, 
20-Hjf, '50+-(X- 2.4, SD- KO) . . , >v 

ESTUSE* Estimated number of hours per week that ttie system will . be 
used, at pre-use; categorized in-six levels (X= 2.26, SDr* .91) 

NO 0NE=» Level of agreement with statement at follow. up that "There Is 
no one on 'this system with wh^ont I wish to communicate a^ great* deal" 
(X= 3.7,, SD= .6)/ • . ' # . 

'TERM- Level/ of agreement , at follow 'up that inconvenient access to 

terminal .decreases use (X s1 2.3, SD-..-9) 

' •' ' 

KNOWN 3 i Pre-use response, "How many members of your EIES group do you 
know eijther professionally or personally?" (X^14.9, 'S.D.-18.4} 

OTHACT- 'Level of agreement at "fjollow* ,up ; on importance of "Other 
professional activities mu^t take higher priority" (X- 1.7, SD-.7) 

tvx - OP CASES- 65***»* * * 

9- 

£ ' ^ CORRELATION MATR^C (PEARSON'S) ^ 

• TERM KNOWN ESTUSE NO ONE OTHACT _ ' 

LEVEL v > .25 .26 .46 .29 * .02 

TERM • .26' .21 .09- .05 

KNOWN/ ■ .26 .10 .10 

ESTUgE ' .15 .07 

NO ONE j .09 



FACTOR 

ESTUSE 

NO ONE 

TERM 

KNOW 

OTHACT 



STEPWISE MULTIPLE REGRESSION 



MULT RR SQUARE 
.46 .21 




BETA 

■.38 
.21 
• .12 
■ ...11 

-.04 



Step 5 P-4.8, p« <iJ01 
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Table 2-17 . 

• • • 

•> .Stepwise Multiple Regression 

- , . * 

Determinants of "Number of Jlours of Use at Follow-Up 

. A • 

■ CORRELATION MATRIX (PEARSON'S) 

TERM ' KNOWN ESTUSE MG ONE OTHACT 

HOURS .31 .48 .46 .22 .08 * . 

TERM- .26 .21 .09 .05 

KNOWN ' • ' .26 .10 '.10 

ESTUSE ' .1.5 .07 

OTHACT - • - .09 

HOURS- Number of Hours on Line at Follow up (X=30, SD=37.8) 

r * " 

,N of .cases* 65 

▼ 

See Preceeding T*able for Other -Variable "Definitions 

. STEPWISE MULTIPLE REGRESSION 



FACTOR 


MULT RR 


SQUARE 


BETA 


KNOWN 


.48 


* .23 


.35 


ESTUSE 


• .59 


.35- . 


• .32 


TERM 


.61. 


•37 . 


.14 


NO ONE 


.62 


.38 ' 


.13 



Step 4 p- 9.3, P= <.Q1 
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.Limitations 

The fact that no observed relationship cfccurs for some variables 

which might be expected to be related tb amount of use of the' system 

does not mean that they definitely will not affect system acceptance. 

Pot* 7 some, variables,- such as sex, age, and education, we do not have 

enough subjects ■ across/ the range of categories tq permit any •' v 

significant differences to easily emerge.* For others, our indicators 

may be »poor or may be important only within, the Context of othej? 

group or individual- characteristics. An' example of the latter is 

that, although we found no, overall significance for* typing -speed, 

typing ability was found to affect system use by Kerr (1980), whose - 

/ 

Whit^ Rouse Conference ' group was older arid not * composed of 
scientists; ^and* -to affect 1 * those with a negative perception of the 
system in Edward '.s- study. . * 

Implications • v 



We have seen that the-strongest predifctor of 'level oC use of EIES is 

the participant's own estimate of the time that will be spent on 

/ 

line, before- ever using- the system. This result is more of a puzzle 

/ 

to be solved by further research than an answer to the questiop^-of 
determinants of^ use. , One observed correlate ^ is the v nupcer of 
prospective system .members whom one already ' .knows, and thus 
anticipates communicating with. 4 But what other factors account for 
the formation of such pre-uae expectations? ' Did "they hear a 
presentation on the system^ participate in demonstration, read a 
book or article? Do the findings imply that it Is important to 
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, systematically- prient and Inforttf" n&»rs abou^^ system before giving 

« 

♦them a chance to sign on-line? Such questions might be answered with 
v a controlled experiment, in which some gr*oup members are given* a 
*tormal introductory lecture or &$t of general readings, and others" 
receive only documentation- or ■ have^ td use the system "cold." Still 
another "possibility , is the unmeasured factors t>f basic personality or 
work style traits, .or perhaps a * "hunger" for more communications. 
Users do seem to "know" ahead of t,ime ^wh^ther or not they will like 
this form of .communication. * . * * 

/ * ' v 

Thus, whatever _ explains* pre-use Expectations or "receptivity" to this 

form of communication, the practical implications are clear. * If 

prospective conferencing . participan-ts do not Expect to use the system 

very much, it is probably a waste* of 'resources to try. to put* them on 

line. Perhaps CC is like sex^in this regard: you 'en jay it a lot} 

more if you really want it befor^you get it, rather than- having it 



thrust upon" y^>u. 



\ 
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APPENDIX TO CHAPTER TWO 



CROSS TABULATION'S OP REASONS LIMITING USE BY 'ACTUAL AMOUNT OP USE 

: K Importance of Red Notebook Documentatiofr 
' ■ \ In Limiting Use of EIES, 

by Time on Line in, June, 1978 -or* at Pollipw Up 



'umulative 
Hours 

<5 * 
5-19 . 
20-49 ' 
50-99 
100+ 

Total " 



Very.- 
Important 



"3* 
2% 

, 5% 
13% 

0 



Somewhat 
Important 



24 
42 
20 
13 
0 
27 



Not 



N 



•Important 




68 


75 


56 


57 


75 


40 


73 


15 


100% ' 


1 8 


68 ■ 


195 



» Gamma = -.13 . 

Chi Square=16.2, p=.04 

/ Importance of Terminal Access 
ixn Limiting Use of EIES, 
by Time on Line -In June., 1978 or atl Follow Up 



umulatlve 
Hours 

<5 
5-19 

20-49 , - 

50-99 

100+ 




Very , 
Important 



•192 
19* 



Somewhat 



Not 



Important 


Important 


26 


55 


12 


69 


20 


1 55 


7 


87 


0 


87 



25% 
~ 7% 

■ 13% 



Gamma = -. 15 
Chi Square»11.8,p=. 16 
Source: Follow-Up Questionnaire 
All Groups . 

N in all categories Is same as -above, for this 
and, subsequent ' tables 
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Importance of System. Being Too Complicated 
In Limiting Use of EIES, 
By Time, on Line in June/' 1978 or at Follow Up 



c 


Very 


Somewhat ■ - 


vNot 


umulative 


Important 


• Important 


Important 


Hours 




i 




<5 


7* 


13 


80 


5-19 


' 17?- 


37 


46 


20-49" 


5? 


33 


62 


50-99 


■ 0 


-33? 


67 


100+ 


0 


0 


100? 



"Gamma* .12 
Chi Square' = ^25.5, p=.£01 
•Source^/Follow-Up Questionnaire 
(11 Groups 



• Trouble with Telephone 
. ' In Limiting Use of EIES, 
3y Time on Line in June, 1978 or at Follow Up 



umulative 
Hours 

<5 
5-19 
20-49 

50-99 
100+ 



Very Somewhat Not 

Important Important Important 



11? 
12? 
1QZ- 
0 

0 1 



8 

15 
25 
2715 
50? 



81 
72 
■65 
73 
50 



Gamma* .20 
Chi Square»15.*4,p«..05 ' 
Source: Ppllow-Up Questionnaire 
All Groups -. 
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umulative 
Hours • , 



Importance of Telenet Problems 
V - In Limiting Use -of EIES, 
By Time on Line in June, 197b or at Follow Up 



Very 
Important, 



Somewhat 
Important 



Not 
Important 



(Dropouts ) 
5-19 - 
20^9 
5-0-99 
100+- 



umulative* - 
Hours 

<5 
5-19 
20-49 * 
50-99 
100+ 



7% 


13 


80 


22% 


22 


55 


26% 


15 ! 


& 


2Q% 


• • 33 


47 


13% 


50 


38 




Gamma 3 .31 





Chi 'SqtTare= 19.0,p*.01 
Source: Follow-Up Questionnaire 
All Groups ' 



Importance of Cost of Telephone-Telenet 
In Limiting Use of EIES,, 
By Time on Line in June, 1978 or at Follow Up 




Somewhat 
rtant 



Not 
Important 



12 ' 


81 * 


7 


80 


15 


80 


20 


• 73 


0 


75. 



Gamma 3 .0b 
Chi Square* 7.82 , ,p-.45 
ce: Follow Up Questionnaire 
All Groups (N»195) 



umulatlve 
Hours 



Importance -of 1 Having 'a Bad Experience 
In Limiting Use of EIES, 
By Time on .Line in' June, 197b or at Follow 'Up 

Very Somewhat' Not 

Important Important Important 



<5 ■ 
5-19 
20-49 
50-99 
100+ - 



9J5 
1636 

5% 
20 % 

0 



40 



' 55 
37 s 



Gamma*, . 29 
Chi Square»24.-3,P !S .002 
Source: Pollow-Up Questionnaire 
All Groups.- 



• / • 
C / 

simulative 
'Hours 

<5 • 

.5-19 
20-49 
5Cf-99 
100+ 



Importance of Limited ?M Hours 
In Limiting Use of' EIES,' 
By Time on Line .in 'June, . 197# or at Follow Up 



) 



Somewhat 



*Not 



ant 


Important- 

* 


Important 


12* 




70 


19%) 




"61 


33* 


t> 


47 


335 




3S 


0 




62 



Gamma 3 .27 
. Chi 3quare»d5.3,P".05 
Source: Follow-Up Questionnaire 
All Groups 
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umulatlve 
Hours 



•Importance of Not Knowing How To Type' 
' . " In •Limiting Use of EIES, 
By Time on Line "in June, 1978 or at Follow Up 



Very 
Important 



Somewhat 
. Important 



Not 
Important 



<5 
5-19 
20-^9 ' ' 
50-99 ' 
100+ 



7%' 

4? 

8? 

' 0 
0 



10 
17 
22 
7% 
25% 



«83 

79 
70 

93 4 
75 1 



Gamma = .08 
Chi Square=6^^,p=.53 
Source: Follow-Up*<ti§estionnatre 
All Groups (n- 195) 



Importance of Not Liking System Like This 
In Limiting Use of EISS, 
By T4.me on Line' in June, 1978 or at Follow Up 



c 


f Very 


Somewhat. 


Not 


umulative 


Important 


Important 


• Important 


Hours 






i 


<5 


4Z v 


8 


88 


5-19 


b% 


7 


87 


20-49 


O 


10% 


^90 


50-99 


-0 


0 


'• /100* 


100+ - 


0 


1355 


/ 87 






Gamma 


» -.15 



Chi Square»5.00,p».75 
Source: Follow-Up Questionnaire 
— All Groups 
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Importance of No One On SysteiirTo Communicate' with 
- In Limiting Use of EIES, 

By Time on Line in June, 1978 or at Follow Op 

C Very Somewhat: Not 

umulative Important Important ' Important 

Hours * ' -s*. 

'I . 

<5 12% , 21 67 

5-19- ' 5% 12' ' 83 

20-49 3* V 17 80 

50-99 . 0 < 1% 93 

100+ / • 0 ^ '0 100* ' 

. Gamma* $40 
Chi. Square»;2. 3,P=. 13 
Source: Follow-Up Questionnaire 
All Groups 

Importance of Lack of Interest in Subjects 
In Limiting, Use of EIES, 
3y Time on Line in June, 1-978 or at follow Up 



c 4 


Very 


Somewhat 


Not 


umulative 


Important 


Important . 


Impbrtant 


Hours 








<5 


9% 


12 


79 


5-19 ' 


5% 


' 20 


75 


20-49 


2% 


20 


78 


50-99 


0 


• 27* 


73 


100+ 


0 


13? 


87 






Gamma 


-.02 



Chi Squar*e*6. 43,P*.6 
Source: Pollow-Up Questionnaire 
^ All Groups 
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Importance of Priority of Professional Activities 
In Limiting Use of. EIES, 
■3y Time on Line in June, 1978 or at 'Hollow Up 



umulative 
Hours 



Very 

• Important 



Somewhat 
Important 



Not 
Important 



<5 
5-19 
20-49 
50-99 
100+ * 



55? 


21 


24 


517. 


• 30 


19 


43 % 


35 


22 


275 • 


47 


27 


135 


62 . ; 


25 




. Gamma* -. 17 





16 



Chi Square=11.7,P a 
Source: Follow-Up Questionnaire 
All Groups ;< 



t 



* Importance of Material Mot Worth' Reading 
In Limiting Use of EIES, 
By Time on Line in June, 1978 or at Follow. [Up 



c 


Very 


Somewhat 


Not 


umulative 


Important 


Important 


Important 


Hours 








<5 


105 


« 20 


70 


5 -.19 


... 95 


« 43 


48 


2<V49 


0 


• 405 


60 


50-99 


75 


27 


66 


100+ 


, 0 


135 


87 




7 ' 


Gamma 


= .02 



Chi-Square=15.3,P=.05 
Source: Pollow-Up Questionnaire 
All Groups 
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Importance of Inadequate Leadership 
In Limiting Use of EIES, 
By Time on Line in June, 1978 'or at Follow Up 



C 

umu^ative 
Hours 



Very 
Important 



Somewhat - ' Not 
Important Important 



<5* 
5,-19 ' 
-20-99 
100+ 



4* 
8*' 

0 

M 



12 
18 
27? 
0 



8.4 
74 
73 
86 



Gamma* .14 
Chi Square*6.52^p=.b 
Source: Pollow-Up Questionnaire 
All Groups 
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Chapter. 3 

VARIATIONS AMONG" THE SCIENTIFIC SPECIALTY GROUPS J 

Among the Variables which h^ve been found to be; consistently 
associated with reported perarfept ions of the characteristics of EIES 
as a communication^ medium and its effectiveness i$ the specific group 
to which a u&er -belongs. In this chapter^ selected findings will *be_ 
presented which -illustrate .that ar^ electronic information exchange 
system is to some extent 3. very pliable or amorphous form of* 
communication and information exchange; perception of the system 
varies according* to the p use 'made 5f it by the members of a specific 
user group. • 
« / 

We will^attempt to»see if any group characteristics are strongly 
associated with the success of a £roup in using the EIES system*" Our 
procedure will, be to roughly rank the groups ^from more to less 
successrul. Then? we will look at somfe characteristics* of the various 
s cient ific user groups, and see to what extent variations in these 
characteristics might be associated with differences in the level of 
success. A section which has the theme "The System Is as the User 
Group Does" follows. It shows how the same system. is characterized 

, r 

or rated 'very differently according # to the group membership of the 
rater. Finally, we will note that considerable "electronic 
migration" occurred among group's by the end of the operational 
trials, so that group differjtfHfcetf began to blur. 

A Note on the Composition* of EIES Groups 

v 

The EIES user groups are not necessarily "groups" or "communities" in 
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the sociological sense of Having dense sociometric .ti*$, nor did 'the 
operational .trials, groups Correspond- to the c<>re 'members of a 
scientific specialty* One .could Jistvg created such on«-lijie "invisible 

i f " 4 

colleges" by starting with citation in^lysis, locating- the* leading 
atuthors in a specialty and asking them to noi&inate^otherf members en 
the basis of their desire to communicate and work together . - • Instead*, 
a single scientist applied to the National Science Foundation, and 
nominated , proposed participants whom he or she kne>w to be wopking in 

the area* The group leaders (principal investigators) were not, 

£ & X ' 

necessarily among the best known or leading or best liked scientists 

within the specialty. As. '.nil be seen in detail in Table 3-2 below, 
even according to probably over-generous self-ratings m of relative 
status" within the specialty , u only a quarter of the. participants' felt 
that they were at or near the top of their specialties. In terms :of 
people they' ranked as major or outstanding in the^specialty , most 
were not on EIES (see Appendix, Preuse questionnaire, p. A 5). And 
the scientists themselves" describe their "groups" as more of "a 
collection of individuals" or "a set of cliques", rather* pnan "a well 
integrated Research community", even at tjie time of follow-up. 



So, in sum, we have collections of scientists-' working in the same 
specialty area, most of whom did not know one another before EIES 
use, ratfler than true "groups" of scientists. 



VARSfolONS IN OVpAl^ DEGREE OP SUCCESS 



There are many ways of measuring the -success of a computerized 
conferencing, effort for the members of, a user group. We might have 
gathered a behavioral measure consisting of the extent to which the 
members felt*- so strongly about the value T>f the system that they 
actively proselytized to bring new members onto the system. We might 
^liave ) counted the total volume of material they, created and read* Any* 
one or two indicators are not valid in accurately judging the success 
of a specific group. Rather, we wish to use a measure that will 
^o'ughlj^ank order the . various groups as more or les^ successful in 



theifr use^of EIES • 



We will use two measures . of ^g^pcess" of the operational trials 
activities * for group members. Qne la the proportion of group members 
who spent enough hours* on . line that one can presume that they were 
participating in' some activity th&t they, felt to be valuable. The 
second la subjective ratings of the productiveness or value of the 
system by those who used it fairly - actively Since most of our 
subjective Ratings of characteristic^ of the EIES . system *eneg 
\ " included on the follow-Aip questionnaire, we want ^frby use hours^ on l£ns \ 
me/wure^s f rop the same point in time. - n ^ 



f 



# * Lookinflf^f irs ft t at hours oh lln$, * it will^e remembered that at the 
three^six months follow-up point;; J:he proportion of groufc member a who 



haci *pent 20 or jmore hours on line .varied as f 



? allows : 
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Group Group # ' X 20+ Rank 

Futures - * 30 46* 1 

Social Networks 35 43? 2 

* ' * 

General Systems 40 4 3^% 3 * 
'Hepatitis 80 30* 4 

Information Sci 50 . 25* '5**^ ' 

« • 

Devices 45 17% 6 

f 

f 

• Mental Workload 54 10% 7 

We will use as an Indicator of the subjectively rated value of EIES 
the c?ean rating of group members at follow up of how "productive" or 
"unproductive" ; the system was. This* was rated on a seven-point 
semantic ^differential scale (1* productive; 7 x unproductive) . ~ 

GrciUp Mean Rank v^*- 



45 


2.90 


1 


30 * 


2.95 


2 ' 


40 


3.. 24 


3 ■ 


[80] 


3.33 


4 


35 . 


3.60 


• 5" 


54 


4.12 


6 


* 50 


t K< 40 


7 



\ 

F-2.2, p- .06 



\ 



A technical note first — Oroup 80 (Hep^frHls) Is shown in parentheses 
because although the same question was used 4 Vt -was not administered 
,on the y standard follow-up questionnaire. Whenever ^HPdata are, 

, - j >~ : ■ - ' 
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available for this 'group, it will W shown in parentheses, implying 
we can make some inferences about tile* relative ranking of group^80, 
but^ that the datfiL are somewhat different. The group 80 data are ^not 
included in 'statistical tests K of group differences. In the above 

c * s 

table, the w P n ratio indicates that an analysis .of* variance was used 
to test significance, and the differences among the groups are just 
short of the rigorous .05 level. 

» » 
Combining the two kinds of information, we can roughly rank order the 

groups as follows; | 



Most successful experiences with EIES* groups 30 ( Futures-me^n rank 
1.5) and 40 (General Systems- mean rank =3). 



Middle level- groups 35 (Social Networks)/ 45 (Devices), and^80 
(Hepatitis* (mean ranks 3.5, 3*5, and 4.0) 

<* 

Least succes^ul- ' groups # 50 ( Jnf oririation Science) W 54 (Mental 
Workload) (mean ranks 6 and 6.5) J \ • ' " * 
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VARIATIONS IN CHARACTERISTICS OP THE SCIENTIFIC- COMMUNITIES 



•I 



Ther pre-use questionnaire ^included many itfems. that could be 
aggregated to characterize the scientific user groups rather th^n* 
Just individual respondents.* It waST hypothesized, that these 
characteristic?/ might be important correlates of sty re and amount of 
use of the .system*, and of the' outcome of the EIES experience for the' 

« L 

grpups that used it* * k * 

* * - 

First, some variables that might be expected to correlate with group, 
but -which in fact* did not* We have seen that both expected use of 
the system before ever signing on and the number of group- members, 
known bef ore>> use of EIES correlates ^ighly with one of ouf components 
of group success i time „<6n line. However, there is no significant- 
relationship between thtfSe Important variables, and group. The' 
differences xhich do exist among the grougs^ln the number members 

,who knew each other bef o^e using EIES are hot significant. The 
groups in which there were the largest, proportions -members knowing 
one another ~b6fore Msfe of EIES were in fact not among the mosft 

-'successful — 35 and 541- Looking, at expected use of the system before 
signing onto EIES, the only clear difference is* between group 54 
(Mental Workload) and fhe others. Over - half " of } Its members 
anticipated signing on less thart once a' week, according to their 

, responses' on the pre-u^e questionnaire. Thus, the 'least successful 
of the five ^operati^npl'k rials groups on, which we have complete data 
started out the -most negative. 



Table 3-1 shows ;th^t none of" the scientific communities using EIES 



had a universally agreed upon "mainstream" or "paradigm"' in Kuhn'.s' 
terminology. There is no apparent relationship within the ranges 

observed between the degree? towhich there was a mainstream and the 

* 

group's uafe of and ructions to EIES. 

X « 

\ 

Table 3-2 shows in more defcifl a point already covered lrr the 

* m - 

f " . 

previous chapter. The groups did differ in terms of the proportion 
of relatively well-known scientists, and the lar'ger proportions of 



eric 



well-known members occured In the more successful groups. Group 30 
clear/y had the largest proportion of relatively well-known members, 
while the least successful group (54) had the fewest (only 14^. 

« 

(Group 50 data are missing for this question and - all pre-use 
measureg). 

Table 3-3 shows the perceived amount and type of competitiveness, by 
group. There are same clear differences In the amount of perceived 
competitiveness. Though there is not, a one-to-one correlation, the 
two most successful, groups had the* largest proportions of members 
perc'elvlug low .or non-existent J overall competition. In terms of 
types of competition, there -is a suggestion that fear of ^methlcal 
behavior among one's peers was moW prevalent in the least successful 
^roup. However, the number of respondents was so v small that the 
differences cannot be considered significant for that question. 
Taken together, ijowever, lbw or nonexistent competitive pressures 
In terms of perception of intense competition and trust that one's 
colleagues wil^not compete unethically are supportive of the success" 
of a computerised conferencing user group. On the other hand, 
perceived competition on. the basis of opposing views or theoretical 
paradigms seems to be- hekl^y for computerized conferencing groups; 



J 



t i 
groups 30 and 40 had the highest reports of this form of competition, 



and the least successful, group t had no reports, of Intellectual 
competition at pre-use. 
I 



ha 
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Table - 3-1 
• • * 

IS THERE AN INTELLECTUAL * MAINSTREAM 

9 

BY SPECIALTY GROUP 



Group 



% yes N responding 



30 


.54* 


13 


35 


27% 


22 


.40 / 


33^ 


30 


45./ " 


71V 


14 


■54/ 


\ 50% 


. 6 



Chi s,quare» 8.7, p*.07 
Source: Pre^Ose Questionnaire Question: Is -there a commonly accepted 
"intellectual mainst ream" in the specialty? 
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Table 3-2 



J 



How -Well Known Participants Were In 



\ 

n Tie 



ir Specialty Areas 4 



i 


-30 


35 

■ 


40. • 


'45 • 


54 


All 


1 (practically 


145 


9* 


.23%- 1 


1555 


it it 


lb% 


unknown) * 




- 










2 


^ 7% . 


4* 


20% 


155 


14 % 


13% 


3 


7 


Q 

y 


T 




0 


10 


4 (average) 


7 


22 


17 


1=5 


29 


17' 


5 


21 


35 






29 


18' 


6 


• 21 


13 


17 


8 


. 14 


15 


7 (top*) 


21 


9 


7 


15 


0 


10 














N responding 


14 


11 


.9 


15 


7 


65 



Source: Pre-Use Questionnaire 

! 

Question: How well known i^your work, within yoyr specialty area 
: 1 : 2 : 3 : 4 : 5 : 6 C : 7 . : 



Practically 
Unknown 



Average 



Ranked at 
Top of field 



i3« 



.116 



r 



. GROC? 

1 30 
35 
40 

4 5 . 
TOTAL 



' NJ. Table 3-3 

PERCEIVED DEGREE OP COMPETITION BY GROUPS 



♦ VERY INTENSE , MODERATE 
OR INTENSE ' 



9X 



242 



43X ( 

OX 

2.U 



55X 

57X 

39X 

50X 

100X 

51X 



LOW OR NUMBER, 
NON-EXISTENT RESPONDING 



36X 
19X 

* 

45X 
7X 
OX 
28X 



11 
21 
31 
14 
5 

o2 



r 



Chi square ■ 16.6, p « .03 
Source: Pre-Use Questionnaire • 
Question: Hjow would you rate the degree or intensity of 

• - * « 

competition within your specialty? 
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' *Tabl» 3-4 \ 

1 X CHECKING SPECIFIC* REASON POR COMPETITION BY- GROUP 

*• • * 



GROUP ' FUNDS . RIVALS 
30(N-10) 18* . . "OS 



DRIVE UNETHICAL OPPOSING 

50* 10* 50* ' 



- 35(N-20) , 21* 
40(N«25 32* 



55* 



20* 



65* 



44* 



5* 



30* 



36* 



' 45CN-13) 21* 



5MN-5) 9* 



31* 



20* 



61* 



15% 
40S 



8* 



0* 



Chi square 2.5 



.64 



11.9 '*> ' 4.5 



.01 



.33 



7.2 
.12 



7.7 
.10 



Source:- Pre-Use Questionnaires 

< 

Number Responding * 73 



Question: What are the reasons for this competition? (Check all 
apply.) * 

Scarcity or competition for funds 
* n Rival groups of idollaborators * 

High achievement or success drivj| of people in f irfld 
Some persons act ^unethically 
Strongly opposing views *r 



that 
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Moras and Counter-Noras In the Scientific Communities 



The norms of Science are 'supposed to stress ^emotional neutrality and 

th$ irrelevance of personal attributes in Judging scientific work 

< 

(See J^rt^i, 1973) • ' Th at . a such a scientific ethici exists has been 
V challenged by Mitroff (1974*). Working with - tfitroff, two sets of 
questions were designed to ' test the perceptions of scientists about 
the fundamental value commitments which characterize their scientific 
specialties. 



Tables^ 3-5 through 3-8 indicate conquerable prevalence of the 

"count e^norms^^Md also some differences a.mong specialties* 

Although the number of respondents to the question i&v small, the 

futurists are unanimous in their opinion that* emotional commitment to 

W v 

^5 one's own .ideas is characteristic of work in this f-ield. The 
' / ^ / . 

specialty in which there are the fewest members believing intensely 

» in their own ideas rather than maintaining neutrality until 

, hypotheses are proven is the Devices for the Handicapped area. Even 

here, commitment Is Judged much more frequent Wian neutrality. The 

two* groups which seem to have been the most successful also have the 

clearest majorities characterizing the£r peers as emotionally 

committed rather" than neutral (affectivity vs*. affective neutrality, 



in Parson's termi.) 



When "asked about their own behavior, the results swing a little more 
towards neutrality (27% reporting neutrality as more characteristic, 
23% saying both govern equally and 5056^reporting commitment to be 
foofe characteristic of their work). Moving on to which principle 



"-ought to" govern 'work in the specialty, we- have a total of 31% ' 
' saying' neutrality, 30% both equally or neither and 39% commitment. 
Examined according to scientific &roup, the patterns of difference 
^/T^r which principle "ought to govern" behavior are similar to those 
which are reported to* actually govern behavior, but; the differences 
are riot as significant (p»,15). Some of the futurists (145) say that 
neutrality ought to govern behavior, but the majority (54/5) say 
commitment ought to-. 



X 



\ 



Looking at table 3-6, we see that . the EIES t scientific communities 
believe that personal' attributes, are taken into account in judging 
scientific .work in their field: "particularism" rather than 
"universalism" reigns. The only exception is Group 35, social 
networks theory, where opinion is ihore evenly divided* In terms of 
actual reported behavior', there are some significant differences 
among groups (Table 3-7)/ with groups 30 and 45 haying the most 
individuals * who say ' that they ^personally use particularistic 

standards in judging the work of others.* 

* f • 

t 

An interesting lack 'of . consensus appears in Table 3-8 • The^members 
of the various scientific user, communities cannot agree on whether or 
'when their relatively new field: Largest proportidn feeling that It 
is still not a recognized specialty. Grdups 40 (General Systems 
Theory) and 54 (Mental Workload, or human' factors more generally) are 
the .only specialties An which a clear majority feels that they have 
been recognized tor at least* a decade. ' * * ' 
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Table 3-9' presents another piece' of evidence about, the relatively 
"unformed" nature of the scientific communities which . used^IES. 
None of them' are felt by. their members to be an integrated research 
community, but. are rather described as just a "collection m of 
individuals" or a set of ^cliques. \ * 

Whether the prevalence of the "counter-norms" held by these 
scientists can, be accounted for by the relative 'newness and lack jrf a 
mainstream intellectual tradition,, or whether 4^he counter-norms might- 



b€ equally prevalent in older, more established specialties, is an 
Interesting -question that cannot be answered with the data from this 
study. ^ 

* 

The final- table in this series V 3-10 ) shows that with the exception 
Of Groupj 35, most of the EIa£ users did not 4 choose to'work in 
relatively well established research areas, but wanted, the risk and 
excitement of working in a n,ew area. The mere successful groups seem 
to have more members who - spurn traditional or established areas of 
scientific work. 



\ 
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Table 3-5 



G roup 



WHETHER EMOTIONAL NEUTRALITY OR EMOTIONAL COMMITMENT 
GOVERNS BEHAVIOR OP SCIENTISTS, BY SPECIALTY GROUP 

Equal 




Emotional 
Neutrality More 



Emotional 
Commitment More 



30(M-13) 
35CN-20)- 



40(11- 3D \, 
45(N-14) 



0% . 

'< [ ' 

30% 



23% 
21% 



\.0% 



20*- 



13% 
36? 



100% 
503 

43J5 



54(N-7) »• 



Total (N-85) 



29% 




143 



1ZS 



Chi square ■ 13.7 p = .08 



• • < -\ 
Source: Pre-Use Questionnaire 



Question: General PrLnciples of'Sciertce* 



Described below 



57% 



t2% 



_. ( are two sets of * conflicting general- principles 

which can -guide the, condu<^-a«4 evaluation of scientific research; 
Please read each set of principles with your specialty area in mind.. 



Principle A. Emotionin_ Neutrality 



Scientists must 
tte^ir ideas .if they 
proved valid. C 
impartial frame of 
the results and Be 




be emotionally neutral and impartial t/owarc 
are to stand a fair chance of ultimately/ being 
oQdjuctlhg an investigation with anything lesk than/an 
irid^ poses the danger .that the sclentist^rt^J^olas 
unable to give up the hypotheses whjrfi Ittety are 



mi 
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indeed false. 



Principle B< Emotional Commitment 



Scientists "must 1>e emotionally committed to , their, ideas if they 
are to stand a falV chance of ultimately being* proved valid. 0U«8 
a scientist believes intensely in his or her own ideas arid ^loes 
everything legitimately in his - power to verify them, there is the 
danger tha£ he Will give up on^-Ms *ideas too ' quickly. Initial 
.inconclusijfe signs of negative evidence do* not warrant a reordination 
of research efforts. ~ The scientist must believ« # in himself and his * 
oim fihdings with great ^anvictl^on, 

Ms * ' * a 

Onjthe basis df your own experience and observations, to wh^t 
extentjdoes each of jJthe" principles tend to govern the everyday ^ 
working behavior of Wost scientists of your specialty? (Please circle 
one "number.) „ - 



A 

Signif- 
icantly 
More 
Than B 



A 

Moder . 

More •« 
Than B "' 



Both 
Equally 




B 

Signif- 
icantly 
More 
Than A 



it 




5 



to 1 and 2, 4 and 5 were combined. 



1 ^ " 



r 
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• ' . Table 3-6- . 

THE RELEVANCE OP PERSONAL ATTRIBUTES TO MOST SCIENTISTS 
•IN SPECIALTY BY GROUP , 



Group 



. 30(N-130 

L < ' • 

^35(N-20) 



Irrelevancy 



31* 



} 



not 



Equal 

15* ~> 
■ 15% 



Relevancy 

5**% : 
$5%' 



40(N-3"1> 

c 

4^(N-14) 

54(N-7) 
Total 

"Ought to" 



35J5 
14% 



29? 



32? 

V* 

5*3* 



10* 
14* 
14* 
13* 
23* 

Chi square * 3.4 p ■ .9 



55* 

t 

71* 
57* 
55* 
24* 



Source: Pr^-Ujse Questionnaire 
Question: 

T Pfincifple C: The Irrelevancy of Personal Attributes 

* » 
The 4 personal* attrftrcrfc^s of a scientist are completely irrelevant in 
judging results and clain^s to knowledge. Each, ciaim in scifence is 
judged impartially on itV^ own merits, by its ability to stand up to 
rational, • empirical test procedures* without reference _to the 
particular scientist. , ^ 



Principle D: The Relevancy of Personal Attribtites 



The personal attributes of a scientist are highly relevant in judging 
results and claims to knowledge. In reality the work of some 



4rs ' 
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scientists ia given credence over that of others. It is necessary to 
know the personal characteristics, background and * motivations if a 
scientist before one -can properly evaluate his or her work. ^ 

' ' \ / ■" < " ^ 

As above, we wish you to indicate * the extent to which t>ese two 
principles tend to govern the everyday working behavior of most 
scientists in your specialty; tend to) govern your own everyday 
Nrorking^ behavior; and ought to. govern the behavior of scientists in* 
your specialty. * * * 




Table 3-7 • 

•whether, personal -attributes are relevant or irrelevant 
to one's own. Judgement op scientific res^jl^s, by 'specialty, group 



^ Group 



30(N-14) 

35(N»23) 

40(N»3D 
45(N«14) 

54(N=7) 



Irrelevancy 

-29*' 
56%- 

45% ' 



7% 



57% 



Equally 
14)8 

oi 

t!9% 



21% 



14? 



^Chl square * 20.6 p a ,06 {uncollra/sed categories) 



Question: See preceedlng .tab3^e 




Relevancy 



57% 



23'% 
7 If. 



29% 



4 
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Table 3-8 

REPORTED, NUMBER OP YEARS SINCE RESEARCH AREA BECAME RECOGNIZED, 
/ ■ BY SPECIALTY GROUP 

• * 

Group ' ' Not yet Less than 5-9 years 10-19 20+ years 

5 years 
recognized years ago 



30(N»13) • 23X " 0% 23% 5«f. 0% 

f 



35(N»21) 14* 33* ' 29? . 5% 19 



40(N»28) . 4* 295 U 18% - Tit 

45CN-12) O^S 5% 58? 17? 8* 

« • 

5MN-6) ^ 17? 19% 17% 50% 17% 

TotaHN-80) 10? 13* 22? 23? t - 32? 



Chi square « 60.7 p ■ .00 

Source: Pre-Use Questionnaire 
Question: What is the approximate year in which your specialty became 
recognized (or will become recognized) as a separate and distinct f 
research'area? 
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Table 3 -9 

Cohesiv^ness of Research Coranuinity by Group 



GROUP Individuals 



Number 

Cliques Integrated Responding 



30 


35* 


50* 


• 15 


20 


35 


33* 




0 


24 


40 


52? 


45 


O 


29 


45. 


30? 


45 - 


20 


20 


50 


60? 


20 


20 


5 


54 


25? 


63 ' 


13 


8 



Chi square* 17-6 p.- .29 
Source-- Follow-Up Questionnaire 

Question: At the 'present time, which of the following best describes 
your EIES group? 

-more a collection of individuals than a research community 

-a set of cliques or subgroups with interests and activities in 

common, but not an integrated community , . 
-a well integrated resea^nbommunity that shares many 
interests and activities \n common 




Table 3-10 

PREFERENCE FOR WORKING IN ESTABLISHED RESEARCH AREAS, 

BY 'SPECIALTY GROUP I 



Group ""ATgree Neither Agree Disagree- ^ Strongly 

nor C Disagree 

Disagree 



^ 30(n»16) 6% ' 6* * 56% k 31% 

♦ 

-~35(n-23) 50% 23% 27% '20% 

e 

40(n-32) 0% 38% 56% - 6%- - . 

• 45(N-14) 0% 43% 50% . 7% 

54(N-7) 0% ^ 71% 29% 0% 

Total 2% 34% 53% ,11? 



I Chi squa ne »_) 18.7 p - . 10 

Source: Pre-Use Questionnaire 

Queitiorf: I prefer to work In well-established research areas. 
(Sti*Q2gly agree, agrde, neither agree nor disagree, disagree, stongly 
disagree. Note that no EIES members strongly agreed.) 
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TOTAL GROUP CONFERENCE ACTIVITY AND CONTRIBUTIONS BY THE GROUP LEADERS 

Table 3-11 shows 'data on the main group conferences for each group.* 
(Group ' 50 f s conf*rence was erased after completion and could not be 
analyzed. )* We see that the f groups varied widely In terms of the 
-amount of activity In their main group .conference. Group 30, Futures 
research, was" by fair the . largest conference, and attracted many 
members" out-side of Its original members, once It became one of the 

X 

* 

^most active -conferences oh the EIES system. The group leader was 

; 

extremely active, contributing more than 300 comments to the 
conference over the course of the discuss^pn. 

The second most active conference was 40 (General Systems), .according 
to measures— tjhe "total number of members, total comments 

written, and number of comments contributed by the group leader. 



Thus, the most sucessfui groups are the ories which had the*most 
,active conferences. Though group members do many things on line 
• besides participate In the common group conference, Its 'success seems 
central to the perceived quality of the whole experience oo line, 

* 

.Group 35 (Social Networks) Is something of a special case. After 
approximately a yeaf* of general discussion, about half of the group 
membership was purged, 'and a second, task-oriented Conference was, 
begun. It is the data for the secohd of group 35 f s conferences that 
is still available for analysis. Group 54 (Mental Workload) had the 
smallest, least active conference, especially considering the size of 
.^its membership. It also had the fewest comments contributed, by a 



j 
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group lead 



e'r. 



1 " 

Group 80 (Hepatitis) is*a.lso a special case. This was an explicitly 
w egalitarian task-oriented conference, in which each person had a 

0 

• \ 

• "designate'*! part 'of the division of latxor. Thus, the nominal leader of 
the expert 'group contributed only 13* of the items (but 21% ' of the 
total lines written; he tended to have longer comments than the 
average member). 



Many fact/brs # *rfay account for these variations in activity and 

J 

apparent success of the main group conferences. One' is the extent, to 

* - '» » , * / 

which -the conference Was able to be focussed on some topics that were 

interesting* ahd Important -to the group members. A second, related 

factor is the level of effort -and skill of the group leader. In 

observing the conferences from week to week, it could be spen that if 

a group leader went on vacation or otherwise disappeared for more 

than a .week at a' time, the conference activity tended to become 

disorganized and then drop off sharply. The group' conferences needed 

a strong, active leader to keep, the discussion organized and m'ovlng 

in, a way that was satisfying to the participants. Table 3^12 shows 

an almost perfect -rank order correlation between the leader 1 s effort 

as measured by time' on line and our measures of the overall success 

4 

of the group. 

Leadership could* be 'split between two persons. For example', group 80 
had a "content" leader who was an expert /on the subject being 
discussed and a "process" leader. Leadership could "also rotate. 
For instance, after the official end of the operational trials, when 
the leader of group* 30 became much less active, conference leadership 
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shifted to another member df the group. Variations in levels and 

• • • 

proportions ol^. participation in conferences will be treated in much j 
more detafcl in ^^port baSed ' on extensive analyses * of the EIES 
monitor data vTuroff and Hiltz, forthcoming). 

* 4 f * 

The Role of a Conference Lea'der 



One of f the "latecomers" to the operational trials established and led 
a futures-oriented conf ereftcT~which attracted mthy members of groups 
30 and 40 and other participants after 1 the. official end of th-e - 



Operational Trials. He T has documented the role played by a 

\ 

consclenclous conference leader/'and the kinds of activities which 
account s for the large proportion of comments contributed by the 

leader Ifi most sucessful 'conferences (Caldwell, 1981). Some of his 

i * 
generalizable descriptions of ^ the leadership -role, are excerpted 

> 

below: ft 

* , 
The role of a moderator is similar to 'that of a committee 

chairman in face-Jto*-face meetings while allowing for the 

* additional unique computer conferencing processes and while not 

having to -wor^y abou<t some of the meeting 1 Chirac teri3 tics which 

relate tp physical presence. 

. .At the minimum the moderator must enter new members into 
the conference (once that is done conference members may delete 
themselves). However, the full responsibilities of a moderator 
ahpuld involve* riore,-than this but will- vary by -1 ^conference 
membership, subject matter, conference activity level, and the / 
personal style bf the moderator and members. . . . 

The conference mo£Ufl"*ator needs to provide some 'number of 
comments 4 which are\jpurely administ rative (as opposed to serving 
as a member of thW^ponf erence if desired). While the moderator 
s.hould take the responsibility of making 1 decisions about the 
conference management it seems reasonable to expect some form of 
member feedback to assist that decision making. In addition, --it 
may prove helpful to' inse'rt certain comments which provide data 
or literature references on topics relevant to ' the conference 
^ discussion (this could be done by any member). . . . 

After the conference had been operating for approximately 30 
comments, it was concluded (initiated by member feedback) that. 

h {* r 

w * 
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an index of every 15 Items -could help tie things together.. In 
addition, a monthly progress report was provided. ' After about 50 
comments were, entered, it seemed advisable to begin an overall 
index 'where both old and 'new conference members could find the , 
other Indices. In this overall index 'there were itemized lists 
of the separate indices pf each 15 or 30 comments • summaries as 
well as locations of the monthly-'prOgress reports. 

There was also a need for providing directions of conference 
techniques to several of the members because the backgrounds 
ranged from member^ who had started'with EIES several years ago 
and others who had Just Joined. Accordingly, there were special 
comments written to provide instructions on how to vote, how to 
delete comments, "how to set conference markers, how tfo use the 
^associated comment number to advantage, and how .to write special 
commands/ for effeciency of ■ time. A special conference comment 
was written which incorporated many of these suggestions and was 
used to introduce new members to the conference (along with £h*^ 
overall index). Thus, a new member was entered and a message 
■was sent indicating the location of the "hints" comment and^the 
overall index. . . 



"in thi3 conference, the moderator account^ for 425 of all 
comments. Most of the time the purely administrative * comments 
(indices, progress reports, 
20-25? of the total comments 



member feetfB'ack, voting) ran at 



The time devoted to conference management by the mcderator 
depends on the amount of administrative experience, familiarity 
with ,tfte EIES system, and the particular admini'strative'. mode 
required of the conf erence . In this conference, the development 
and entering of the monthly progress report took approximately 
30 minutes afoi the indexing of each 30 comments took about 40 
minute's. 0#ne\al ■ evaluation of activities and reflections on how 
to modify conference directions took an estimate.d three hours 
oer month. 



The level of activity by the Sroup 45, lea4er«2orroborates Caldwell's 
obligation that process-oriented activity by a consciences 
conference moderator may account for 20-255 of the total number of 
items. Jane McCarroll, the' moderator/leader for Sroup 45, points out 
that unlike the other group leaders, fshe was not herself a member of 
the scientific community whose activity she facilitated. That is, 
she was not herself engaged in the development or testing of devices 
for the handicapped, though she was familiar with the area. Thus, 
her contributions were by definition mostly process oriented rather 
than substantive; yet they took 22% of the items in the conference. 
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Table 3-11 

i 

Group Conference Activity 



Conf Members Comments # by 



t by 









Leader 

» 


Leader 


30 


• 61 


1278 


312 


24* 


4 


OO 

22 


Qrt 

289 


33 


11% 


40 w 


45 


389 


73 


19* 


k • 


34 


- 237 


52 


22* 


54 


23 


1 138 


30 


22* 


80 


ii~ 


265 


33 


13* 



N i 



- J 



13* X 



X 



Source: Monitor Statlst\cs^^^^ 
Table 3-12 



CUMULATIVE TIME ON LINE BY- GROUP ( LEADER AS OP APRIL 3, 1980 



Group 



30 
40 

45 
35 
80 
54 



Total 
Hours By 
Leader 
' 765 
755 
557 
320 

293 
^29 



Leader Group . 
Hours .Rank Success 

Rank 



1 
2 

3 
4 

5 
6 



1 
2 

3.5 
3.5 
5 
6 



r 
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Ratings of the Group Leader 

% 

/ 

0 . . 

The only group where- a significant portion of the members listed 
inadequate leadership as a reasori which dissuaded thein from usin£ 
EIES was Grpup 54. Five of sixteen group' 54 respondents- to this 
question (31*) checked this factor as ^"importanS^ 

In response to a direct question which asked "How would you rate the 
performance of your group leader (principle investigator)" 



excellent 



poor 



only groups 50 and 54 have a. significant proportion rating the leader 
below n 2, n *as shown in table 3-13 • Although there are other factors 
which also contributed to the 'lack of success' of these two" groups, 
relatively, inactive leaders does seem to be an important factor. 



\ 



ERLC 



135 



143 <\ 



f\' Table 3-13 

i Ratings of Group Leader, by Group 
Excellent 2 5^5 N responding 



55%^ 25% 20% — 20~ 

W" 30% - 22% 23 

tk% 27% ^ ll* 26 

* 

b% 2bJ m 17% lb 

4 60* 40* 5 



f 



J»3* ^ 28* 29 



Source: Follow-Up Quest tonrwti^. 
Chi square * 14.9 p * .78 (not significant) 




VARIATIONS'-^ SUBJECTIVE REACTIONS TO- THE SYSTEM, BY SPECIALTY GROUP 

j • Ob 

In this'/Vs^^on, we wj.ll lbpk at some examples^ of way^ In which the 
~~s ame^- ob^^rclv 9fbtp pabl 1 1 1 e s 4 or qualities of the ^ETES^systefe' are 
di*ff essentially pencelv-ed and rated, depending on the group content In 

which -a person uses ^the system. First* we will exajnine the reported 

,; 

responsiveness oS* people to electronic messages on EIES, which Is one 

measure- 6f the effectiveness of the message component of> th« EIES 

. . . . * ' r< * 

^ystem.. 

* v 

The messfge ^^jj^ett'^p designed" as a replacement a for letters and 
% telephone caTls. Of course, It Is effective oniy If members sign' on 

* to .receive twelr messages regularly, ^ and ^answer them rather than 

* • ' * ■ 

ignore them. 1 
.J } 

- / 
Generally, Table 3-14 shows, fhat the electronic message Is seen as 

more effective than or equally effective as ,*a telephone call or 

letter. It^should be noted that the two groups for ~ which perceived 

effectiveness- is not particularly good* are tlie smallest. It has 

previously been hypothesized (Hlltz and Turoff, 1978) that there Is a 

critical mass phenomenon. There' must " be a* large ertough^fc umber of 

active members of at user 'group who siga on -dally or almost every day 

to generate % *^tt*e motivations for all members to sign on frequ ently 

and to eritel^coinmunlcaf^ons into th6 system. Othgpwlse, the patt< 

of dally sign on whltlflg Is nepes*sary for such a system to be 

efficient (speed£), communication system Is negatively rewarded by "'no 

~~ new items, waiting 11 when a member signs in, afid members are 

discouraged. . * •< 




• Ta^ble 3-14 _ ' 

Responsiveness to EXES messages, by Oroup 

• ■ 



More 



GROUP 


Responsive . 


.Less 


No Diff. 


N 


30 


55* 


20 


J. 25 ' 


20 


35 


42* 


. 1.2 


46. 


24 


40 


66* 


20 


44 


25 


. 45 


40* 


• 35 . 


. . • 25 


20 


50 


0 


80 


20 


5 


54 


142 • 


43 


43 


7 


' all 


39* 


26 


33 • 





% 

' Chi Square* 21.2 p* .13 .'" ^ 
Source: Follow-Up Questionnaire 
Question: When you send a" message over EIES 'rather than writing 
telephoning, would you say" that recipients are generally 
-More responsive to an EISS message JT 
-Less resjuonsive ^* • 

-No difference 
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Group 



Table 3-15 - 
HOW OFTEN USERS PEEL DISTRACTED BY THE MECHANICS 
OP. THE EIES SYSTEM, BY GROUP 



Always or Sometimes • Almost 
Almost Always Never 



Never 



- y30(N-21) 

/ _ 

35(N-25) 
■40(N-28) 
45(N»21) 
50(N-5') 




bor 



54(N-8) ^635 
Total(N-loa) 215 



485 
605 
465 

405 
125 
495 



195 

285 

325 

145 

05' 

255 

235 



55 

405 
75 



05 



05 



65 



Source: Follow-Up Questionnaire 

Chi square » 34.1 p - .63 

\ 



13-9 



15 



'A 




There were consistent though not always statistically significant 
differences in most other' perceived characteristics of the EIES 
system associated with group. Table 3-15 indicates that only the 
members .o^ fehe user groups that" had the least successful field trials 
on the system t tended to frequently feel "distractedly the mechanics 
' of the system." Whether EIES is, stimulating or boring, -frustrating or 
not frustrating,' is '"also greatly influenced' by 'group membership 
(Tables 3-16 and 3-17). Groups 30, 40 and 45 'are consistently the 
most positive; groups 50 and 5 1 * consistently the most negative* 



Tatfle 3-X6 

EIES Is. . .Stimulating '- Boring 

. , (Seven-Poiirf Scale) 

( L- Stimulating 7- Boring ) 



roup' 


1-2. 


3 


' ■ ' 4 


5-6 Total N 


'Mean 


30 


» ..65* 


26 


. 0 


9 


23 


2.22, 




56* 


24 




» 4 ' 


25 


.^.56 


40 


59* 


34 


0 


7 


29 


2.38 J) 


45 


62* 


14 


. 19 


"5 


, 21 ' 


2>43 1 


50 


40* 


20 


0 ' 


40 


5 


3.40 


54 


25* 


50 


0 


25 ' 


a 


3.13 


C80J 


5b* 


17 


17 


8 


12' 


2.61 

- - 4- 




Analysis of 


Variance 


p- i.4d 


p- .23 






Chi Square 


(uncol lapsed 


data)- 45.6 

* 




0007 
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.Table 3-17 
Whether EIES Is ."Frustrating" by Group 

Seven-Point Scale 
( 1» Not Frustrating 7* Frustrating ) 



GROUP 


1-2 


3 


4 


5 


6-7 


N 


X 


30 


in 


22 


13 


35 


13 


23 


*4.oi» 


• 35 


- 20* 


12 . 


28 


20 


20 


25 


4.12 


40 


28X 


31 


21 • 


17 


3 


29 


3.34 


45 


19* 


24 


24 


29 


5 


21 


3.57 


50 


0 


40 


20 




40 


5 


4.40 


54 


0 


12 


25 


25 


38 


8 


4.88 



Source: Follow-Up Questionnaires 
X}hl 3q«are (•uncellapaed da-ta)- » -4.28 •« P* 
Analysis of-, Variance F- 1.88 P- .10 
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Blurring Among Groups 



AVt^ie Operational Trials proceeded, many members of th^ various 



scientific communities officially Joined other groups afid conferences 
i£ addition to their own.' ,In the table below, one 3ees, for 



/ 



end of 



instance, that of the 24. members of Group 30 on line at th< 

/the operational triads, four were also members of Group 35 i eight of-, 

r Group 40, etc. One has to total the numbers above and to/ the right 

to read the entire table. For example, of the 66 members / of Group 

40, seven' were also in Group 35 and five were also in Group 45* 

^Mi^S^tlon 4nto two ot % tJfxg groups not Included in ^ Jh^s study^ (60, 

JEDEC, and 70, ¥ LEGITECH) - are shown for completeness. Groups v 30 and 

40 had attracted the most cross-memberships with other groups, , and 

Group 54, the least. By this point, many nominal members of the 

various scientific communities were actually spending more of their 

i 

■on line time participating in another, group 1 s conferencing activity 
than in their own. 
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Table 3-18 
NUMBER OP COMMON MEMBERS BETWEEN GROUPS 
April 1, 1980 



0 


30 


35 


40 


*5 


54 


60' 


70 


80 


30 


24 


4 


8 


3 


1 


1. 


5 


1 


35 




25 


7 


' 2 


0 


1 


4 


1 


40 






66 


5 


0 


1 


9 


1 


45 








30 


0 


0 


5 


1 


54 










21 


0 


0 


0 


60 












50 


•1 


0 


70 






> 








36 
* 


1 


80. 
















16 



Source: EIES Monitor data 
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Summary 

4 

« * . . 

1. The. scientific user'groups on EIES 'were collections of. individuals" 
and cliques' in the same research specialities, rather than "groups" 
in the sociological sense* 

* 

2. A relative group success index was generated using a combination 
of the proportion of members who spent at least 20 hours on tine, and 
subjective satisfaction with productivity gains as a result *of using 
the system. Group characteristics were compared to relative success. 
We find that: ^ 

/ 

a) The .research specialties represented' on EIES do not have an agreed, 
upon "mainstream" or paradigm" of theoretical and methodological 
principles. Within the range of , low to medium paradigm groups 
observed, there is no relationship between extent of paradigm and 
relative success. \ / 



b) Intellectual competition within a specialty appears to stimulate 
use of the system; other types of competition may hinder it.. 

* * 

c) fir^ terras -of the pattern variable emotional committment 
("aff activity") vs. neutrality, the more successful groups tiave the 
highest proportion of . members who tend to believe intensely in their 

theories, rather than maintaining neutrality until hypotheses are 

) . ~~ 

proven. 
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All of the groups tended toward particularism" rather than 
'"universalism. " ' . 

. f. ' " * • 

d) Most EIES scientific users prefer to work in new research areas 
rather than in well established are§,s; the more sucessfui groups 
tended to have the largest proportions of would-be pioneers in new 
research areas. a 

e) The amount -of on-line activity by the reader seeing to be strongly 

related to the. success of ttue group. 7 
> - * 

3) 1 Ther'e kre correlations between the overall "success" of it group 1 s 
-efforts on line, and the -subjective impressions- of the system formed 
by group members. For example, the least successful groups, are most 
likely to feel "distracted by the mechanics" of using the medium, and 
to find the system itself to[ be "boring" and "frustrating." ■ 

4) Although users typically Join the system is members of a specific 
group, as they gain experience they tend to communicate with members 

I; 

of other groups, too, and 'to Join other conferences. Initially 
distinct user % groups become overlapping networks. Interesting, well 
led conferences* thrive and grow, attracting memfoega, f#om other 
groups, While other conferences essentially grow moribund' and the 
group members stop communicating with one another on a group 1 basis. 



User groups within conferencing systems might be compared to 

/ / r ' 

subcultures witjiin a society. Being a ' member of one group 
(subculture) rather than another seems to shape the experiences of 
the members and the quality of their (electronic) life. 

% 
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Chapter Pour 
THE EVOLUTION QP USER BEHAVIOR 
(Coathored with Murray Turoff) , 

/ - ■ ; 

TWs chapter examines changes in the behavior and attitudes of users 
in relation to specific features of the system, changes which have 
some design -and policy implications. There ace many other aspects of _ 
'changes . in the behavior and ^ttltudes of both individual users and 
user groups over time which are not treated here, such as bhanges in 
perceptions ^ o*f the usefulness of the system, for various 
communications function (see Cltepter five), subtle changes in the 
style and richness of written communications (see Carey, 1980, for a 
description of; paralinguistic behavior), asd changes in the social 
organization and productivity of user groups (see Chapters six and 
seven). | - 

INTRODUCTION j 

1 • ■ . / 

Since-^the earliest observations, those who. Joave ' studied 

'- 

comptuter-based qommunication have recognized' that, as Johans/en 

/ . * 

(1976) states, "initial uses of teleconferencing systems often serve 
as a poor basis for generalizing about future uses." The datar~ from 
this study provide, for the first time, detailed empirical evidence 
about changes in user behavior and preferences related to the 
features or capabilities of com{5 , Huteribased communication systems, as 
a function of experience -(hours on line). They will also serve to 
show that amount of experience is a powerful determinant of many 
aspects of user reactions to systems such as EIES. As a result, all 
variables on all questionnaires were cross-tabulated by hours of 



experience, and are reported* in subsequent chapter whenever they are 
significant. 

* * 

Limitations of the Data , 



The reported result^ are limited to a single system and a single type 
of user (Scientists). Until similar measures are replicated for 
other systems and ot £gi" 1 ifff es of*, users, the generallzablllty of the 
specl-flc results obtained for EIES is unknown. Another limitation is 
that the data currently available for analysis are cross-sectional 
(attitudes and behavior measured at a single point in time) rather 
than longitudinal studies which measure " each user's .amount of 
experience and opinions about the system at many points in time. (1) 



(1) We attempted a longitudinal analysis , but did not have enough 
cases in the critical' ranges for proper analysis. Thje 'number of 
cases for which we have knswers on the same question on the value of 
features on a first follow-up questionnaire at approximately six 
months after starting to use EIES and on the eighteen-month post-use 
questionnaire ranges from 55 to 71. However, a total" of only twenty 
were in the range which evolved from fairly new users to experienced 
users during this time period. Regression analysis and Pearson's 
correlations on the relationship between change in hours on line and 
change in ratings of features showed relationships that were 
generally in the predicted ' direction, but were not statistically 
significant. we think that the fairly weak relationships are due to 
the/ inability to* capture measures on the users at critical points in 
their learning behavior when relying on two questionnaires a year 
apart, and Have chosen not to report this analysis. 



However., the basic generalization to be drawn from the dataA that 

* \ 

t&ere is Indeed .an "evolution"* or pattern of change towards greater 
complexity and specialization and "diversity of user behavior oyer 

i - I 

tine, is consistent enough with studies of other teleconferencing 
systems th^t it * is not likely to.be an artifact of the limitations of 
this study. (See Elton, 197^, and Johansen, Vallee and Spangler,« 
1979:13b-137 for similar generalizations jbased upon other 
teleconferencing systems). 

, * . <~ * * 

§ 

1 




Background: The' Proliferation of Simple* Electronic Mail Systems ' 

Compute'rs are' increasingly being put to wor|c 'in the processing, 
.storage, 'and transmission of text to facilitate human communications. 

The most, widespread proliferation 'is\takfng place lr; the 'areas of 

pi 

. "electronic mail" and "word processing." Uhlig (19.7-7) cbmes to the 
. same kind of optimistic bonclusion \about the future Importance of 1 
electronic ,mai^ as do* the" majori'ty of those Who have studied, this 
\ m technology: 

During the next 50 years computer bashed message systems " 
> (CBMS's) will have as great an impact on the way business 

is done in -our society as the impact that tJie telephone had 
on business* practices .during the last 100 ye^rs. This, at^ 
least, is what our organLzat^on-*has come tb believe after* 
two and one-h4lf years of experimenting with them. _ 



Electronic* mail is usually designed with a minimal number of . 
features, so that it can simply replicate electronically the delivery 

» 

of ''mall" ^nd internal memoranda. For example, this limited set - ( of 
functions is implicit in .the recent paper on the design objectives of 

0 

message systems by Levin and Schroeder . (1979: 29) that refers to 
"Message systems that communicate memoranda among members of a 
community." Word processors are likewise, b$irrg desigried as 
specialized, single purpose systems, to be used ohly by secretaries, 
acting as Intermediaries between the originators ar\d , the recipients 
of text. 



Ir^hls review of "The Outlook for Computer Mail", Pank,o *( 1977) 
concluded: * \_ 

Computer .mail has.atgreat deal going for it; apparently 
# favorable economics, ' a huge' potential market, and weakening - 
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postal opposit^n. To tap- this rftarkeft ,\a fair amount of 
design evolutioiFwill be required,** 



^We agree that v "'design evolution" will be necessary .in orde^r to 
• A maximize the role of the computer in the* facilitation' of human 
-v communication. ?urthermdreir we believe that such evolution -should J?e 



based upon feedback from th^ experiences of users, of current jyst 



• 



ft 



> m 




V 



• 



9 

ERIC 



151 



165 




EVOLUTION OP USER BEHAVIOR 



After approximately eighteen months of use of the JJflfc system, 
members, of yie scientific user groups' on line were asked to rate the 
perceived usefulness of a number of specific system features. t{ 
they had not us$d a feature at all, they were, instructed to check 
"Cannot Say ; " otherwise, they were to rate each one as "Extreme'ly 
Valuable,"- "Fairly Useful," "Slightly Useful" or "Useless." 



The . data in Tables 4-1 and 4*E show the relationship between amount 
of .time, spent on line and rating's of the usefulness of the system 
features. -Let us look at Table 4-1 first. The first column served 
as thl basis for ordering the features, and is simply the proportion 
of the total of 102»users answering these questions *N<ho rated a 
feature as, "extremely valuable." The responses at the\other end of 
the scale, "Useless" and "Cannot Say," have been combined^fc form a 
more nearly -drdinal scale, since v£T*y few checked "useless." "Cannot 
say"* yas the response that was checked 'by respondents -who felt so 
little need for the feature - that they did not ever try to use it. 
-Some" ""of this is accounted for by poor documentation of the newest r of 
the features, which are not included in the user manual. 

^ v A . • - 

Column five of Table 4-1 reports a statistic whic£ shows, the amount 
of ' relatio^^ip between the subjective rating g£ • the value .of the 
feature used, and amount of use of "the system at t^he time the 



questionnaire was written.^ Gamma,., the, statistic used, is a 
correlation" coefficient ,' which varies between -1.00 and +1.00, with 
•' zero meaning nS 'relationship . It is the Inos't /commonly used measure 
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for ordinal scales. It^ Is a "PRE" (Proportional ^Reductior} in Error > 
measure, A gamma of .5 in table 4-1 can be interpreted to mean that 
if you pick any two pairs of observations in 'the sample, it is 50? 
9 more likely that the person who is higher in hours on the system ai*so 
has the-* higher rating for -the fe^ure, 6han that they vary in the 
opposite direction. It can also be interpreted to mean that overall, 
knowledge of time on *£lne improves our prlffiction of syatem feature 
rating by 505. \ (See Crittenden and Montgomery (1980/ for a 
discussion ' of measures of associat ion^Sr ordinal variables). 

The last c^umn- shows the level of statistical significance of the 
relationship between time on the system and subjective ratings of the 

V 

value of the features, " based on a Chi square test. We have c^cided 
to; always report results significant at the* fQ5 level, a rough guide ^ 
to the extent to which ttfie observed patterns of association are too 
strong to be at tributed* to rartfom variations associated with sampling 
error. We'.* will usually ueport results significant at- the .10 level, 
if they seem consistent with other findings, and will sometimes 
• reDort findings which are even' more tentative and In need ^of 
replication (findings for "which the probability that the results, 
would not'ihold with a< l&rge sample is greater than 

The most universally appreciated features are the private message, 
the direct text editing necessary to mak'e typing c (fractions, user 
consultants to help one find ohe's way around the'system, and system 
commands to replace a menu-driven interface when users understand the 
options available. These* are the types of features which are built 

T 

systems do not" include ttte "friendly human helpers,." the user 



Into most electronic mail |sys terns, with the exception .that most such 
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consultants. However, high overall popularity ratings are also 
received b^ marry v featur5s whiclj are not usually, part of electronic' 
mall systems: group and private * «*onf erflhces, and the publiG 
directory"^ members to facilitate the formation of Interest gro'ups. 
- In addition,- we notice from gamma statistics that appreciation of 
many features appears to be related significantly to amount of use of 
the system. 

. ■ 

This becomes clearer In Table 4-2* Here we see that beginning users 
do^ indeed see the need for only a relatively small number of features 



.In . a computer-based communication system. However, the more 
experience they gain, th^more. they come to feel that a widq^variety 
of communication, spaces and capabilities 13 necessary, and zhe less 
liVfciy they are to be' satisfied with a simple message system. The 
group-oriented* and conferencing features become much. more important, 

* as do the -features that are necessary for storage,, retrieval, 'and 
manipulation of text for document3. > 
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FEATURE 



*^?able 4-1 

Reactions to Specific Features of the EIES System 
ajnd Correlation (Gamma) with 
** Time-On-Line 



EXTREMELY FAIRLY 
VALUABLE USEFUL 



SlfcGHTLl 



JHTLY^ USELESS, GAMMA 
USEFUL CANNOT 



Private. Messages - , 

Text Editing (direct) 

v'e.g. , /old/new/) 

User Consultants 

System Commands 

(e.g. ,+cnm) 

Group Conferences 

Group Messages l 

The Directory \ • 

Private Confe£ f fc«fces 

Retrieval y/\ 

Searches \ * 

U&er Defined commands 

(i.e. +Def ine) 

Text Editing (indirect) 

x e • g • , • text ) 

♦SEN and ??? 

Chimo 

Private Notebooks 

Use of ?,??' 
-Explanation File- 

Terminal Control' Features 

ve:g./'+left,+page) 
. Anonymity j5v 

Pen Name 

Synchronous Discussion/ 
in Conferences 
Group Notebooks 
Special Programs (e.g. 
♦terms, ^respond 
Graphics Routines 
Interact 
Programming 
Tailored Interfaces* 
(e.g. ,+Legitech) 



Games. 
Voting 



e.g.+story) 



. b8% ■ 


j 22% 


10? 




1 8 

x U 


A 
u 




PI 
<- X 


7 


403 


' 27 


7 




j j 




35% 


31 


< 25 


3*% 


35 


17 


33% > 


25 


, --8 


31% 




■ 9' 


27% 


J 16 


lb 


C I it 


ID 




20% 


16 


3 




2 1 

C X 


10 

X w 


1.7% 


23 


24 


i*% 


23 ' 


7 




25 


16 


1015 


20 


19 


" 1 o" % 1 

X \J /* • 




7 


10? 


13 


16 








9% 


16 


*' 7% 


15 


6 


9% 


.9 


6 


7% 


5 


2 




3 


6 


n% 


6 


3 


.3% 


6 


21 


2% 


12 


7 



SoUrc'e* Post-Use Questionnaires »• JJ-102 



OA X 


K 




1 


.50 


.09 


•25 


„.23 


.47. 


1 22 


r .32 


. -02 


2h 
c U 




01 

. w X 


1 5 
12 


an 


. . 04 


0 




48 




p 1 

• ex 


04 


35 


.44 


- .01 


30. 


••30 


.48* 


38 


.-38 


.01 


5^ 


.29 


. ooi 


ft i 

U X 


1 7" 

. X ( 


. 16- 

• XV 


-.51 


• 58 


-.001 


.36 


• 34 


.20 


r 5 - 




oh i 




,11 


.24 


51 


.00 


.82 


66 


i22 


*V."19 


61 

\J X 


. 32 


. . 25 


63 


1 7 


. D 2 


72 


.03 


• 39 


76 


.40 


' . 12 


86 - 


.42 


.21 


86., 


.20' 


f . 16 


87 ' 


."41 


* ~03 


70 


• 55 


, .002 


79- 


.18 


.15 



•Probability that relationship cpuld.be due to sampling error, Chi 
Square test 
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TABLE 4-2l GROWTH OP FEATURES PERCEIVED AS 
: "EXTREMELY VALUABLE" OR "FAIRLY USEFUL" 

* AS A- FUNCTION OF" AMOUNT OF EXPERIENCE USING EISS 
(* indicates addition to list over prior usage class) 



USERS WITH 1 TO 19 HOURS ON-LINE (N-26) 



FEATURE ' 
PRIVATE MESSAGES 
USER CONSULTANTS 
GROUP MESSAGES 
GROUP CONFERENCES 
DIRECT EDITS 
MEMBERSHIP DIRECTORY 



31 
71 




\ 



63 
59 



/ 



USERS WITH 20 TO 49 HOURS EXPERIENCE (N=32) 



FEATURE 

private messages 
3rcjj? conferences 
direct edits 
System commands* 
vser consultants 

"\ j ROUP MESSAGES 
/RETRIEVAL* 
\/ PRIVATE CONFERENCES* 
' MEMBERSHIP DIRECTORY 

1 J . 



«4 

66 

65 
64 

59 
52 
53 
53 
56 



% SHIFT 
+3 
+3 
+2 
+21 
-11 
-6 
+5 
+ 17 
-3 
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Table 4-2, continued 
USERS. WITH 50 TO 99 HOURS EXPERIENCE (N»25) 



FEATURES 


it 


% SHIFT 


PRIVATE MESSAGES 


96 


+6 


GROUP CONFERENCES 


80 


+14 


SYSTEM COMMANDS . 


75 


+ 11' 


MEMBERSHIP DIRECTORY ' 


72 


. +16 


RETRIEVAL 


68 


+ 15 


USER CONSULTANTS 


67 


- . +7 


DIRECT EDITS 


67 


+ 1 


3 ROUP* MESSAGES * 


54 


-8 


SEARCHES* 


* 52 


+26 


? "AND ??* t ; 


52 


+ 10 


PRIVATE CONFERENCES! 


51 


-2 


3 END , LIN K , A ND ? *? ? * 


50 


+26- 



X 



USERS WITH 100 HOURS AND OVER EXPERIENCE (N*19j 



FEATURE 


• % 


% SHIFT 


PRIVATE MESSAGES 


100 


+4 


MEMBERSHIP DIRECTORY • 


95 


+23 


USER CONSULTANTS 




+28 


DIRECT EDITS 


•90 


+13 


GROUP CONFERENCES 


90 


+ 10 


SYSTEM COMMANDS 


90 


+ 15 


RETRIEVAL 


•84 


+ 16 


GROUP MESSAGES 


84 


• +30 


PRIVATE NOTEBOOKS* 


74 


+ 4u 


SEN, LINK, AND ??? 


79 


+29 


USER DEFINED COMMANDS* 


68 


+31 


CHIMO* - 


63 


+42 


INDIRECT EDITS* 


6 3 


+34 


PRIVATE CONFERENCES ' 


4* 55 


+ 4 


TERMINAL CONTROL* 


53 


+ 46 


Source: Post-Use questionnaire 


and Moni 
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EIES *ls, admittedly, not what It should be In terms of user 
documentation. As an RID system with low levels of operational 
staff, there Is no regGlar^ documer^at Ion effort. New feat&tes arise 
from user feedback via zhe user consultants, and evaluators to ' the 
implement ors. when a new feature Is added, it Is exposed to the user 
consultants, who test It and write documentation *for the /^on-line 
file. Major new features kre announced in CHI.MC, the "Sn-llne 

m 

newsletter. After that, a user must -either search the explanat 

jfile or ask a user consultant If a feature exists to fill a perceived 

"need. There is no regular mailing of updated documentation to 

users. As a result, a user must feel motivated to seek out new 
features and to learn to use them without any f a^e-to-f ace training. 

We tnink tnat the users themselves seeking out new features aft^r 
gaining experience on line^makes our results more significant than 
^they would be if they were simply responding to pushes from Tt aaVanced 
.training • seminars" or published training manuals on the features 
which they "ought" to learn >hen they feel eomf ortable with the basic 
system. 

r 

Although tne likelihood that a person will find a system feature 
necessary or useful Is generally positively correlated with use, 
there are a few exceptions. Some of the f.eatures for whiqh perceived 
usefulness seems to be a direct function of amount of use of the 
system are: gt*oup messages, group conferences,' private conferences, 
3ystera commands (as compared to the menu selection Interface), search 
-routinei, and Indirect editing for formatting of output. 
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r 

Dne interesting drop is' in the perceived value of * group messages, at 
the intermediate levels. We .think that .new users perceive the 
feature from the point of view of the sender: a convenient way to 
communicate with a , large group. With a little more experience, 
however, they become aware of unwanted group messages from the 
recipient's point of view. Group conferences, in which receipt of an 
item is govern.ed by self-selection on the basis of topic/ is then 
, seen as' a\^more valuable, self-filtered mechanism Tor group 

communication, within the context of the EIES design. 

1 ' ■' 

An interesting curvilinear pattern occurs for user consultants; 
appreciation of them is high at all levels, but the newest and the 
most experienced users find them most valuable of all. This is 
probably oecause trie user consultant is asked for help and human 
response ("Somebody talk to me!" )*by ne ophy t e and then becomes the 
source of "advanced knowledge" on features that t are too new or. 

r 

complicated to be automatically retrievable by the shorjt explanation 
request (7 and ??). This tends to occur when users master , the basic 
system and are ready to' move on to preparing large documents in 
notebooks and defining their own commands. 

% 

Another complementary explanation, partially verified by observation, 
^^^is that ths user consultants also take on gatekeeping and information 
brokerage roles. They are often asked by advanced users for 
Information bn whether particular topics might be dfscussed and who 
else 6n the system might be interested in them. In a sense, the user 
consultant' represents a new type ,of numan facilitation role for the 
electronic information exchange^ environment. They also advise on 

' effective' styles of leadership for users who wish to establish a 

/ 

/ 
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conference or other activity on 1 



irje. 



Looking at the pattern of phanges, one can interpret them ds showing 
that new users appreciate a° system that, replaces communication media 
with which they are familiar, / These are the letter and the 
telepnone call (replaced by* the private message), and the meeting 
'.replaced by the gt*ou^ conference). However, as they gain experience 
with the new medium, their perceptions ' of useful applications 'and 
"their preferred styles of using the medium change. 



As users gain more experience with the 'medium, they tend to find more 
valuable . the unique kinds of functions which the computer ^rr^rovide 
for asjjyachronous group efforts. They need features which help them 
to deal Ifcth "lnf ornation overload," which can result from - intensive 
daily interaction with a large number of people and groups. They 
also begin to use other advanced features that can be provided by a 
computerized conferencing system. 



One can classify those features for whicn there is a substantial 
increase in perceived usefulness as a function of experience as 
follows: 

s 

1) Features that facilitate long-term group communication rather than 
one-to-one communication c (the group conference arid the .private 
conference) , 



2) Features that allow a user to actively control 



the s^te 



m rather 



tnan passively react to menu choices and new items automatically 

/ 

presented -(system commands, user-defined commands, .searches). 
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However, It should be noted* that EIES members teel that the menu is < 
the optimal interface for the beginning user. , 

3) Features to support composition and the preparation of larger text 
items and documents (notebooks, indirect' editing, and^ terminal 
controls for formatting output). Note that it is only at 100 hours 
or more ox" experience that most users arrive at the point where- tht^jr 
rfant to produce their large documents on line, rather than having 
them typed. 

/• 

4) Features that. permit tailoring ok the system to -individual and 

« * 

group needs (user-defined. commands, special routines, and the 
INTERACT language,). 

Phases of User 3ehavior ' * 

One classical model of user behavior in interactive systems with 

which one can compare our data was developed by Bennet (1972). He 

generalizes user behavior into the UNCERTAINTY phase, during which 

the "learner has to- overcome hesitancy and anxiety; the INSIGHT phase, - 

during which the user understands the general concept of the system * 

and can make at least limited use of it for his or her own. purposes; 

t 

tae INCORPORATION phase, when the mechanics of the interaction become 
second nature; and the SATURATION • phase where the system is perceived 
as inadequate for ^eeting ng' w req uirements users evolve as a result 
of experience. 

f - . 

EIES risers reDort a median of 1.7 hours to ,learn the' basics, but 
there Is ^quite a wide variation (the mean is 6.4 hours). Reaching 
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the "Insight" phase seems to be related to becoming comfortable with 
the writing style and multi-strandedness of conferences, where) many 
topics tend to bei discussed simultaneously, A median of 4.5 hoCrs /is 
reported to feel comfortable using the system. The "Incorporation" 
phase appears to have occurred by fifty hours. To date, we have rrot 

1 0 

observed any signs of the "Saturation" phase, except >^n the form of a 
desire to learn the INTERACT programming language and ^onstruct one's 
?wn 'subsystems, or have another person do ''the programming to 
specifications of the ulers. 

There / is a phenomenon of "information overload, " which seems to set 
in ; r on all regular users sooner or later. SIES ' Drovides -many 
conferences and activities vrhich users are free to join, far more 
than the number wiUa which any individual can cope. Tl^e growth in 
puolicly available conferences and th3 fact that a new user can go 
back and read a. conference transcript that has been accumulating for 
a/ year or more makes the ' accomulated^iaterial in EIES like -a data 
ase. The % plethora of available material creates a need for 
searches, .r^ettieval, gfnd the ability to select material of interest 
from all that is stored on line. Thi3 overload phase is now 
receiving considerable attention in the evolution of the system 
design. - - - 

\ 

_ Incorporation or Addiction? 

Some of the users' who have spent a- lot of time* on. line and ha\e 
Incorporated EIES into their style of work -refer to themselves —'as 
"addicted," or make comments that could be interpreted as signs "of 
addiction. Some examples of this are: • _ 



4 

^5a 



) 

"I can't think when the system is down."" 



I can live without EIES, but i can T t LIVE without *EIES ! " (conference 
1003, Impacts) 

\ : • J ^ , 

(Vnj^plaining reasons limiting use on the post-use questionnaire,..) 
"Tl\e only pressures were the need to sleeD and to continue the 
obligations of a life that already consumed 16 to 18 hour? a day. 
9ut rpr that I would have signed on EIES regularly 4 once 'a day, for 16 
hours eacn time." 

(In response to post-use questionnaire item on changes in the way one 
wfcrks an3 thinks...) I spend 1-3 hours per day on EIES, usually^ in 
the morning, often on weekends and at night. It has become 
'essential 1 to me." 



"During and after the Berlin WFS*meei>ing I became somewhat addicted 
to EIES." 



A study of "ex-addicts" had been planned as part of the extension for 
this study. The operational trials groups were ended on April 1/ 
1980. The plan was %o study users who had spent more than one 
{ hundred a l^ours on line, at app-roximately one month and six months 
after their, last Use 1 of EIES. "*^wever, practically all of them 

managed to find th£ funds somewhere to continue EIES membership, if. 

« * 

■only as "class 2" members paying out of their own pockets. So we 
have no "cold turlJey" behavior to rep<^t. 
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j i CONCISION 

ST * 

The design implications of thea*<^ observations ire fairly obvious. 
Short-term pretests of inexperienced users on small-scale systems 
cannot be generalized* "b^^pedlct the preferences of experienced users 
•on operation^ ^systems . m 

* * * 

< • 

User^ cannot tell you what they need prior to using this technology. 
Attempts ; to* pre-rdesigi*» fixed systems, which ar'e common in the^ 1 
standard data base area, are doomed to failure, unless the group 
setting the requirements are experienced users* of the technology. 
The difficulty in validating this statement is^ th,at * people in dire 

\ need of improved communicatipns will utilize anything they are given 
which provides increased efficiency. .Simple message systems will do 
this, but they, will, silso leave the user in ign^ance of other 

B opportunities which thl*s technology can offer. 

j In terms of the analytic JLfnpliaat idns for this ~st4^, . our findings 
/ indicate that amount of experience, as measured. by hours on line, is 
likely to be related to many aspects of patterns of use of the system 

\ 

^nd attitudes toward it. Therefore, itr >will be -used as an 
independent or control variable in looking at all of the dependent 
variables for th<is study. 



Perhaps CC systems' are more like wine* than roses., "A role is a rose 
is a ros^..." but experienced users of CC, Hke oenoghiles-, \come" to 
appreciate design subtleties and complexities', and to' want to be able 
to choose Just the right feature to suoDor't| or complement a variety 

of communication activities. r . 

% 



V - 

.'ASPECTS OP SUBJECTIVE SATISFACTION :' LEARNING, USING, AND.. 

REACTING TO EIE5' * . - * w 



« * ♦ 

' CKajhker Fiv.e - • 



•A ' . 



t 




In ttiis^ chapter, we ^rhall examine In considerable detail users' 
opinions and reported ^ex'perl^nces related to EIE$. We. shall* sta.rt ^ 
; with descriptions* of how' long^ it takes y them 'to learn to use it and r 
their -style, of use. Then we shall proceed* to examine "opinions "~ about 
^ancfc reactions to specials features, particularly the interface 'and 
" % ttiQse aspects - of the system that are most and least liked. 

* i ■ 



' Finaily, w,e 'shall look at overall^pvalua'tlons, of the system both in' 
t^mS of global characterizations^ -of it as being "good" or "bad," % 
"stimulating' 1 or ""boring," and in terms of its. perceived utility for 
t ' specific types' of communicat i<?n and infc^matioh exchange functions, 
v A multivariate* % arfjffysis wil^ be* ,used to construct? satisfaction 
iridices'and to, identify the mo§t important determinants of subjective,-, 
.'satisfaction. ^ " . - • < . . 

v . , . v . LEARNING TO' USE THE SYSTEM 



Durirtg the operational trials / ElES Unfortunately combined a complex 

and 'evolving system- with a related lack of -complete, -up-to-date 

"v - \ , - . ' 

documentation and'- learning 'guides. As result, many users felt that 

it took them too long,* even on a -simple level, to learn tp "use,* the' 

system a'nd .that earning the advanced features was toQ mu^i of a time 



Investment. 
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At the follow-up, we a^ked u&grs. how long It took them to ^learn to 
use the system reasonably, well." This is a rather vague, global 
question. _ The reported median Is 3.4'hours, but' \l% report -10-20' 
hours, resulting In a ^nean of 4.96 hours. • 

.At the post-use time; we broke • learhlng down Into three different 
aspects: the basics, feeling comfortable, and learning advanced 
• * features. - ^ .Most report- less than 5 nours to learn th^ ba 
mechanics, though one in five report faking longer. The median is 
2. 4 l -hours. Thece is no relationship between any of the measures of 
previous experience with using a computer or a computer terminal - and 
the time-it takes^ to learn the basics- of EI5S. 

For "feeling 'comfortable" m the median Is 5.1 hours. "Peeling 
comfortable" is yer/ strongly related to time on-line at the time.tjie 

question is asked (a correlation of .53 as measured bv eta, 

\ t * • 

* 

•significant' at tne .01 level). The more time a person had been on 
line, the more time -in retrospect they report it took for them to 
"feel comfortable, but the less likely they were to report that they 
never fe.lt comfortable. Thus, the curvilinear "coefficient (eta) is 
- stronger than the linear one (see Table 5-^). 

Learning *of advanced £eatui*es \s , obviously /y a problem* with the 
system. ^ 'Abdftf half of thos * witless than 50 hours of time on line 
•never 4 learneflL them at all*, and orte-third of the high users with 50-99 
if&urs/Qn line 'did not. The more time a- person spends on line, the 



longer they-report, taking to learn advanced -features. For instance, 

Among jthose with' more than 100 hours on line, over one-third report 

' % < W > 

30 hours or -more fro^leafrn the advanced featur'es (see Table 5-2). The 
" • s* \ > \ j r / 
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difficulty of learning the advanced features and the, associated 
feeling that the system is f too complex 11 also "come out in many of the 
optional, open-ended comnent5 about things liked least about EIES and s 
, about most needed improvements. 

f 

An alternative interpretation is that the system is, "rich 11 , rather 

than "too" complex. ff It is. obvious that users "dd not saturate even 

after 100 nours of experience. The designer points cut that a 

conscious choice is made to let users know that there is an almost 

^ endless array of>advanced features to be learned if they wish to- 

^ learn •hem. 3y the end ^of the operational trials, EIES nad- over 500 

command a. aoove and beyond, tne menu functions^ ^^sev-er^l. 

— ^ . + 
specialized subsystems' and its own programming language that users 

could master. 




Included" in the Appendix are two examples .of^f^porta on th^-^'ues tions 
asKed user consultants by EIES" members. ^ These* show some of the 

0 

poirr^s qf difficulty' encountered in learning to use the system. ^The 



eports aij 



reports also* served* as one . of the main mechanises for ''formative" 
\ 

evaluation, since reported difficulties wec* rs vsed • a3 a basis for 
modifications to the system or its documentation, 

v 




• . .. • ■ -!;. , 

Table 5-1 s . - - 

Hours on LUte to Reach Various Learning Levels 

<2 2 3.1 5 6-10 11-97 Have Not 

3 IX ' 20 27 14. 6 . 2 

Peel comfortable ' : 19% / 18 -15' 14 24 * 10 

Sourde: Post-Use Questionnaire 




Question: Hop many hours do you* feel it took you to, learn the basic 
/ /-\rriech&nlcs of sending, and receiving messages" and comments? ^o, feel 
^comfortable commutiieating with others using this mediCyjtf? 



7 



1 



\ 
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Table 5-2 - . 



Roups Taker^to Learn Advanced Features, 
by Tirrfe on Line 





Cumulative Hours 


on 


Line - 


• 




Hours to 


1-19 


20-49 50 


-99~ 


100+ • 


All 


















<2 


9% 




175 


•' 0 


9% 




2-3 


27 


13 . - 


22 


24 • 


22 


» 


4-10'- 


14-. 


15 * 


9 


0 


10 




11-29' 


4 

* 


7 * 


4 


24 


9 




30+ 




" 0 


17, 


35 


1-1 ■ 




Have Mot 


t 

_4b 

» 


52, v 


30 


« 


3b * 





s 

Question 
advanced 



> t 
Source: ?ost-Use Questionnaire 
Chi square = 37.2, .04 
eta * .5 

*Kow many 'hours do -you .feel It to6k you to learn 
features *that you wanted to use? 



>. 
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Documentation 



New users of EIES are provided with a loose-leaf red binder called 

"How 'to Use EIES. " It covers-all .of thfc • basic features and' also 

- • - j* ^ 

includes # a one-page "users' guide" that is a map 'of, the system and a 

list of 'frequently used editing' symbols, as well' as a "quick start 

guide." "The extremely condensed - one page users guide is included in 

Appendix. It shows the aet ^of "roenues" that are provided as the 

oeginnlng interface -to move around the system in order to create, 

modi£y, send,t receive, and select communications of various types. 

At, the time of £he follow-up, ratings of this documentation are quite- 



satisfactory (see Table 5-3 )• Almost all find it readaole* and fairly 
easy to understand. Ease of understanding is 'positively correlated 
^jjfith accumulated time on line when the question -is answered, but it 
is not possible to untangle the causality here. Does Reeling that it 
is understandable lead to more use of EIES, or does more use of EIES, 



?ad 
ise) 



and therefore more use) of- the documentation, make it seem easier to 
understand? The weakest point is its -.organization, whi.ch one third 
rat$d as neutral or poorer. 



However, this 



^initial documentation does not cover the new ' ^nd 



changing features of EIES that are ervolving^or any of - the advanced 
features. Moreover, a large portion of readers claim that they have 



not and will 
file was cons 



not read through long manuals. An on-line explanation 

i 

;ructe.d to serve as ^.comprehensive, constantly updated 



source of information on all aspects of the system. 
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From the . information received from the follow-up questionnaire during 
the late spring of 1978, we concluded that very little use was being 
made v ^ tWe explanation file. Only 25 rated it "extremely valuable," 
and SOS said they could not say anything about it because they had 
never used it. ' The user consultant file showed that the they 
frequently had to tell people ' ^iow to look things up in the 
exolanation file. 



y was decided to^modularize the on-line .instruction fof the evolving 
system - with the introduction of two new features. ?W0RD (i.e.', 
?message; ?edit) gives a paragraph to one-page explanation of any 
feature on EIES and can be entered at any point. Second, a system of 
one-line explanations invoked by a "?" entered at any point when EIES 
13 ready for input and of half-p^age exolanations invoked by two 
question m^rks ialv linked to all of the choice points. Thus , whenever 
a user dofes not lenow what to do -or what .options are available at any 
.point in IIES, documentation can^tte easily retrieved. 



r 



User acceptance is somewhat *bett>er, with 125 of those in the post-use 
questionnaire rafing this docunventation feature as "extbemel: 




valuabl/e." However, since this .new style of documentat i< 
included in the wrltteft "How to. Use EIES," thosje users who m3>&rrf' have 
the most need for it are, not likely to find out about the new 
docuinjetatatlonf It was ^ig>ained in CHIMC, the on-line ^newsletter, 
which jfas devised as.a means of ' keeping users informed about changes 
and .new features &g well as new groups and activities on line. But 

0 

only \ab out. one-third of , the systei^ members read CHIJIO with^any 
rsg&larity, ' and tJiese'tend to be. hea«{y 4 , users • • « We do not* know the 

ejctent to' which this "?" feature*- might have reduced learning 
• i 
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dlfflcul 
earlier. 



for new and advanced features If It had 'been loanable 

I 




Table 



fe-3 



Ratings <Jf Documentation: 
?ollow;-Up Questionnaire 



^•Co you how .find "How to Use EIES" 



^riders tandable (1) 
2 . 

3 

Confusing (5) I 

Correlation with hours of use (gamma) 



475 
385 
115 
35 
15 



.^7;, 0 = .005 



Easy to read '1) 
2 

3 
4 

Hard to read (5) 



•435 
75 
75 
35 



J 



Correlation wlth-h^ure-nrf-^s-e— f^amraa) = .29; p = .21 

Well organized r ( 1) 255 ' 

2 405 

3 235 • 

4 95 

Not well organized 25 k _ 

(5) . * 

Correlation with hours of use (gamma) = .22; p » '.17 
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Alternative Heip Features 

v " ' . C . 

As a result • of an evolutionary design process, EIES thus offers a 

wide variety of alternative mpans of helping users who ne,ed to learn 

about some aspect of the ' system, in addition to the written 

documentation. Since users are free to choose any combination of the 

available aids,, their relative popularity may be of interest 'to other 

designers of interactive computer systems in 'deciding which types are 

most important to include." f ; * 
* * * 

Table 5-4 shows the reported relative frequency of use of the various 
on-line help aids. Thf most 'popular arW the human "user 
consultants^," described in detail below. Next most widespread use is 
made of tne on-line newsletter ("CHIMO"). This 'is followed fctf the 
"?word n system, and the full -explanation file.' is iea*t used. 
However, all of them are used frequently enough so . tnat ideally, a 
system should incorporate the full range. 



/ 
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Table '5-4 

i 

Percent Making ! Frequent or 'Occasional Use of 
On r Line Help Aids, by Time on Line 



Feature 



•Hours on Line 



User Consultants 
CHIMO- (News) 

? or ?? 
Explanation Pile 
M 




.20 


20-49 


50-99 


100 + 


All 


Gamma 


P 


80 


67 


33 


95 


79 


.26 


.29 


56 


' 67 


87 


89. 


73 


.47 


.02 


43 


50 


62 


' 56 


• 52 


.11 


.55 


70 


' .43 


48 


• 65 


. 47 


.18 


.28 


25 


33 


24 


• 19 


101 







Source: Post-Use Questionnaire 
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>T^e User Consultants - 

The user consultants ,a.ve volunteers who receive accounts and TELENET 

time in^ exchange for 4 playing multiple roles in'serying a^ go-betweens 

£or the system and its users. ^They supplement the printed arid 

on-line documentation' in helping both, new and advanced 'users to learn 

how t'6 i*4e various parts of the system.' They provide a human source^ 

of .support and encouragement / and serve* as peoplebrokers * in assisting 

u§ers in finding others interested - in, the s^ime topics. In addition, 

the user consultants test new features ^nd actually 'write the 

documentat.i^g for theft; these functions .ar*e .generally not visiole to* 

i 

# other EIES members. 

The user consultants are very popular* "As pointed out in Chapter 

Four, in the post-use checklist of the usefulness of various EIES 

* - .* 

features or capabilities, they are ranked near the top at all , levels 

- 

-of -experience* In addition, there was a question on 'the "follow-up 
questionnaire 4 that provided for -open-ended ^ comments about user 
consultants. The^ question read .as 'follows:^ 
Have you ever asked a user consultant fo£j]£l*? 
Mo 

Yes (Please describe whether * this #as helpful,, satisfactory, 
.courteous, or whatever.) A • . 



itely, -th 



Unfortunately, -the question was biased toward -a positive response. 
Most jisers asked at the follow-up point did report contact -with a 
*user consultant (82 of 108" responding answered 7, yes ,f )* Of these 82, 
67 made 1 jfavorable, -comments— but mast took the easy way out of simply 
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circling one or/ more of the adjectives, such as "helpful," 
tf g£tisf actory, " or "courteous." m Those who wrqte something in thei.r 
own words are quoted in Table 5-5 • Though there are a few cases of 
nonresponses from the user consultants or of mixed opinions about 
their* helpfulness^ most of the qomments are quite enthusiastic. 



/ 

\ 
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- Table 5-5 - 

* * • * «, v 

Comments about the User Consultants 

- > •- 

"Excellent and friendly" • 

*"Prompt!" ' . ^ ' 

"Great 11 . " r ~ • . * 

"Very useful" \. • . < 

"Very prompt and useful" * \ m 

•"Very satisfactory, ver'y cour^epus, very erTJ^yable! ! ! " * * 

"Unavailable in most cases" ♦ m " • / 

"Not too rielpful^-merely- repeated what 'I already knpw" 

"Fantastic!" 

"Couldn't get tScough^ 

• "Helpful, mildly courteous, When asked one l^ow to add people to my. 
conference, he answered* that' starting "a conference' was a big deal and 
was. I sure I could handle that-?! ?" . , . ( 



x "Some consultants were helpful;^ others -were not." 
"Nice people" m ^ ^ i 

f, It was- useful." V- / \ ' m ~ 
"OK' — answered ^question" ' * r . * " * 

"Courteous, prompt, usually * but^ hot always 11 ', r • , # 
"They're great-V 0U know that!" * 

"Very helpful, 'quite courte'ous./They are es^entfal^V ^ . ' • 

'"9 times' ourt of 10 the response >is prompt-/ helpful* courteous, and 
friendly*. 4 ■ Occasionally a* re^ues^ seejms to""b£ ignored." ...... 

/^'Yes — bUt?--a bit "more .^kindergarten; -approach "needed"s > ^ ; ' f - - t ' 



"Very helpful,' -sat^sf a'ctary, 



\ and courteous. I am very' J impressed by 



the. services pJbojided^by thes^e people. ". 
, • ff They Wier'e helpful and coartinaius and -answered' rft£ question -quickly* "« 

V 



■ e ,5-5, cont. 

"Yes, very helpful and frienc 

"Courteous but unhelpful. - 1 r^s.'we have had a lot of trouble with IBM, 
^terminals. Good luck. f " 

,» ' • 

"Very helpful. Often my question was in search of information that 

w$s "not really- availably. " 
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• . SYSTEM* INTERFACE ^ 

-. * - 

By "sys.tem interface" /e mean the way ih which the user and the 

system talk to each other — the nature and style of the 'interface, 
s 

the editor, etc.j . As'mentioned in an earlier chapter, EIES ha.s five 
alternative levels' of interface: long menu, short menu, answer ahead, 
commands^ and "self-def iifeci commands. 'However, beneath eacn of these, 
levels lies the same*ba«ic structure 6f prompts, error messages, and 
editing capabilities. 

• 4 

»f ' * 

P ' 

W^used the terjfi ,f system -language" to refer, to the-style of the user 

interface,' the way it 'responds to user actions. This, receives/ 

generally favorable uper ratings. On a five-point scale in which one 

signifies "understandable"* and five M conf using, tf 422 rate it one, 402 

two, 13?, three Uhe neutral point), and only a handful negative. On 

a scale from "courteous" to "inhuman,." results'are almost identical. 

« *■ 
405 give it the\ highest rating, 3M a two,^L55S a three, .neither of 

\ \- * ' 

these variables. is significantly related to time ori line. In fact, 

exactly the same proportion" of * the newest users who answered the 
questions *give the language of the EIES interface the highest rating 

as do total EIES, users.* r 

* * 

OA the pther hand*, there is a strong, significant relationship with 
group 'membership. The negative ratings occur in the groups that are 
most disgruntled* and dissatisfied with EIES as a whole, and least 
"successf ul- f by other measures (Groups 50 and 54': see Table 5-6). 



The editor droes_ rtot>,fare*' so well.. Though the majority of person^ 

* ■ j / .< * t 

give it satisfactory, ratings, a small but substantial proportion 

I ' - % 

dislike JLt very much and rate it* among ,the most -unsatisf act ory things 

about 3IES. In terms of being ''easy to remember" or "hard to 

f • - 

remember," 3235 rate the direct edit symbols (such as /old/new/) as 

one, or "completely easy- to remember," 1635 as two, 21% as three 

^"neutral, neither ea&y,nor hard" ) , as- four, and 10% as five, or 

"hard." This is weakly, Related to time on line (gamma = .36, p = 

.11). It is significantly related to group (Table 5-5). 

In terms of being "easy to use" or "hard to use," the direct edits 
receive similar but 1 slightly more favorable ratings ('3fS*h one, 3^% = 
two, 153 = three, 7% - four, 75 = five). m Ther^e is, once again, a 
we^k, not statistically significant relationship with accumulated 
hours on line. 



Indirect editing commands control text formatting on output, rather 
than being used for correcting mistakes. i For example, the command 
.tabs is used to set up columns in tables, %nd the command .text, 
rJuSt is used to format text so that it skips lines between 
paragraphs and fills text from Margin to margin on the receiving 
terminal, regardless of .how the text is typed in; and right justifies 
or lines up the rignt margin as well as the left. The indirect 
editing commands caused the -most confusion. In terms of ratings, 175 
rate them 'as pne on a scale where one equals "good" and five equals 
"poor;", 173* give them a -two,. 39% a three; 105 a four, and I5**a five, 
or "poor." There is no relationship with* time on line. 



The user consultant file Contain3 many questions from members asking- 
< 1 



about how to use ,the family "of commands under ".text" *that control 
such things # as -indentations H margins , , skipping betweea paragrapns, 
and spacing. These were not irteiuded in the .original written 
documentation because they^-'were developed during the- operational 
trials. Even though a few of ttje # members of EIES* put^ the text editor 
op their list of the? most valuable features, more put it an .their 
list of nominees -for "worst .feature. ff These comments and the group 
membership of those who made them help tp pinpoint tne nature of the 
dissatisfaction. The EIES i editor is .a line-oriented,, editor meant for 
tnose- working on a terminal printing at 30 characters' p^r * second- 

s 

Anyone who is used * to working on a direct-wired CRT at a high baud 
rate finds it m»st slow and clumsy. Though most SIES users do^not, 
fit> into the latter category and many of tnose who are used to high 

speed^ CRT's ^qw have their own micros with \built-in editors, a 

X 

^ossibJLat improvement to EIZS would be to make a'n alternative editor 
available toVhose who are working on CRT's. \ * . 

■ \ • 
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'Table 5-6 

Is the Language of the SIES System Understandably i 



* by Group * v 

1 * 2 3 4-5 • * !J Responding 

(Under-* - v (Cqnf using) / 
standable) 



40* * ' 405 ' 205 Of. • „ ; ' 2|p ^ 

,485. - 40J6 125 ■ 0< 25 

■ 

41* • 48^ . 33 ! ' 75 ' *29 

523 . 335 145 05 - 21 

20% 20% ' 40% 20% , '5 ^ 

t 

•135 3«5 , 12% ' 385 t « ti 

* 

42% ' ' 40% ■ 13%' 5% 108 



Source: ?o.llqw-Up Questionnaire 
Chi square = 38*6, -p = .01 
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Table 5-7 

Are the Direct Edit Symbols Easy to Remember or 
Hard to Remember, by Group • 



1 

Easy 



.5 

Hard 



N Responding 



322 



29.2 



452 
0 2 
0 2 



32% 



23% 
26% 
36% 

20% 
■ \\ 
40 % 

12% 



26% 



IS% 

17% 

25% 

20% 

0% 

38J? 

21% 



142 



42 



112 



152 



202 



0,2 



10! 



142 



92 



02 
02- 
402 
502 
102 J 



22 
23 
23 
20 
5 

a 

106 



i 



Source: Follow-Up Questionnaire 
Chi square =35.4, p = .02 
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.Provision of a Variety of Interfaces 

A design decision was made to provide ,a variety of alternative 
iriterfaces. ^The theory was that they would form a kind, of 
progressj^n, ' with most users* starting out with the long menu, which 
requires no knowledge of the system arid no' membry of its structure 
whatsoever, and progress to tile short menu; from there they would 
begin ^utilizing answer aheads and commands and, finally, frequently 
use their own defihed commands or strings of operations. 

The data show that there is such a pattern. However, they also show 
that there is a great deal of individual variation in' interface 
preferences aijd patterns of use. Although it Ik .true that the long 
menu is the preferred interface for - new users and . becomes less 
frequently used the more experience a person has, its use never stops 
altogether.' Among those with 50 or more cumulative hours on line, 
415 report that they "sometimes" use the long menu. ' Apparently, they 
turn it on when they use new or v unfamiliar parts of the system or 
when they have been away ' frocu the system for a while and^ need to have 
their memories refreshed. (This figure does not appear in Table 5-8 f 
which sbows only the frequency' of **the "frequently" and "often" 
responses.) Thus, though there is' a"\tendency for the predicted 
•progression pattern, one cannot automatically change the interface at 
a certain point in time. After experience is gained, commands are 
, the most frequ-ently used interface, but the others are uaed either 
habitually ■ or from time to time by a large proportion of the system's 
members. 
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User Support for Xearnlng Menus First 



'.Users who have previously used command-drivef^ systems are sometimes, 
impatient with the menu as an introductory interface. However, the 1 

majority of users^ support .the design decision to teach menus first 

» t 
(Table 5-9). ' Using the menu seems to have- the cognitive effect of. . • 

\ ■ ■ 

• helping the user to develop a mental map , of , ,the structure of the 

* • 

•system. When the user understands the sturcture of ^he different 

\ 

parts of the system and the relationship among ;them, the more active 
command mode can be used to move around the systgm at will. Support 5 
of the menu as a beginning interface grows stronger the more time 
person Spends on line. ' 

Forced Delivery of Messages. 

%l somewhat more controversial aspect of the EISS design is that, % 
although users may postrpcne delivery of messages, such undelivered 
Messages will remain i"n the queue,* and the user will be frequently 
"notified of their pending statu? until they are accepted. - Some users 
wish to be able to reject the delivery of messages without ^finting ' 
them out, perhaps on the basis of author or key^s. Forced 'delivery is 
not made .of items in conferences or notebooks, where members are free 
to read a header only, >the full text, or nothing. 

The designer's point of view is that confidence that k message^ sent 
will actually- be delivered Is more important than\ the temporary 
inconvenience* of a recipient. Furthermore, it is Wgued tha£, if a 
person is sendingl overly wordy or irrelevant • messages , other^^rbup ° 
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« members- should 1st him or her know, rather than surreptlclpusly 1 
refuse delivery of further messages* from the person, A particularly- 
sticky "design argument la' what to'"do ab^ut confirmations if rejection 
of messages .were Indeed permitted. Since delivery of all messages Is 
normally confirmed, should a comparable notice of a rejection of a 
, message be returned to an author? * 



In looking over suggestions from users in the user consultant file, • 
the most popular design alternatives . sugg£stfd are 1 either makirfg 

• ' 9 * 

acceptance of group messages (but not of private messages) optional, 
• in which case; authors could at any point check a cpnfirmation list if 

they want ,to ' know who has actually read a message, or allowing' 
1 rejection of any message yith some sort of notification, 'For all 
users, the modal preference is support of the current design, with 
forced (eventual) acceptanc^ of all messages. This is endorsed by 
half the members responding overall, and the support of the design 
V decision increased with experience (see Table 5-10) • The second most 
popular' option, endorsed by a quarter of/users, ' is to allow rejection 
of any message, with some sort >of notification to an author 
available. Many people sugges\ some kinder term than '"rejected* or~' 
"refused, " such as "NAME has .been notified of pending M###*."| »An4 
/ about 4Z suggest some other alternative altogether. Thus,* th^fce is - 
no. one solution to this problem that will satisfy everypne, -.but ' the 
forced delivery of at ieast private messages is generally endors$*L 
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Table 5-*8 



ernatlve Interfaces, .by Time ,on Line: 
> * 

Percentage 'Using 'Interface ^Frequently" or "Often 



. ? : 
> 


\ Hours< 


* an Line 


• 






• ♦ 


<20 


20-49 50 


-99 


100 + 


. All 


Long menu 




36% 


0 


0 


33% 


Short menu « 


* 41% 
« 


40% • 


69% 


50% 


44%^ 


Answer ahead* - 








75% 


39% 


Commands 


. 34% > 


71* 


74% 


75% 


48% 


String , 


0 


336 


43% 


-25% 


10% 


Variables ^ 


i 










N responding 


' 41 


35 


•12 • 


8 


- 96 



Source: Pollow-Up Questionnaires 
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Table 5-9 ' 
Preference for Teaching of Menus or CQfomands^ First , 

by Time on Line 



tlenus First Commands First Other 



5-19 hours 5255 
* 

« 

20-49 hours > 74£ 

50-59 hours 83* 

100+ hours 88* 

Total* „ .7055 



3855 
22!l 

, 8% 
■'■12% 
2H% 



10% 



11% 



$%■ 



6% . 



N 
29 
29 
12 

8 

78 



Source: Pollow-Up Questionnaire 

Question: Do you now think It Is -'a good Idea or a^p-oor Idea to 
Introduce the new user to the system through menus and^to • provide 
^fulvalent commands for those who prefer them? 
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Table' 5-1*0 • 

• * 

Percentage of Users Faitfiring the Requirement 'that All^Messages 

r * 

Must Be Accepted' by Addressees, by Time on Line 
Cumulative Houi^s. % 



<20 hours 
2h-l»9 hours 
<N 50-99 hours 



t 



100+ hours 



43% 

51% 
58% 
71% 



All 



N = 103 



50% 



.Source: Follow-Up Questionnaire 

'Question: In ETIES, you do not have the choice of permanently refusing 
•to accept a private 'or gVoup message. Which of the following would 



you prefer? 



Require acceptance of all messages, as at present. - 
Require acceptance of private messages only. - * 

Allow rejection of any message, with "message refused by 4##" . 
returned to- the -sender. Comments? • - .**'.. 
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- ' . % ' ' DEALING WI*H THE PAPER MONSTER % . 

.The massive amounts of- ffetper generated by % a medium* that is supposed 
♦ 

to* be the precusor of a "paperless society" is the .subject of* mufch 
Joking k aryi^ of" ' genuine distress/ In the public^bopf ereqpe on 
."Impacts, " for instance, trfere.are mentions of - having to buy more* andL" 
larger waste baskets and of taking out t6e garbage more / frequently,* 
The long 'roll? of thermal paper'on the' portable terminals provided to 

many users are especially difficult to store, since they are *not 

• „ * • . * • 

pecforatfed and -do not easily fold or divide into pages. 

=: •-• 

A's shown Lin Table . 5-11,.. users vary -in how they handle their 
printouts. Some develop complex * indSkihg . an<3 filing systems, 
complete with .color 'coding. A Pew go so far as >to* keep- written logs 
of ^11 messages sent, dates of confirmations , -etc. Only a few throw 
awa^the printouts* The modal -method i# to establish %^tegories by 
conference or group * rfmnber and to file hard copies, within these 
, categories, thus simplifying retrieval and review. 
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0 " Table 5-11 

Disposition of Printouts, by Group 



* 


•' 30 ' 


35 


'40 * 


45 


54 


Total . 


• 

Throw 'all out 


0* 


■8* 


0* 


0* 13* 


ii at 

4* 


a ( 

-Keep them all * 
Save selective 


.9* 

32* . 


32% 

f 

12% 


10* 


'24* 
U9* 


25*' 


18* 

.23* 


entries irt singles- 






t 


9 






file 

Save selective 


• ' 32% 

9 


t 

• 28* 


45* 


3855 


25* 


J 34* 


entries in separate 

J 














files* 

Use a CRT - 

/ ' 

Other 


. .23? 


< 

- 12*^ 
^8* • 

> 


* 7* 
7* 


14* 
14* 


38* 
38* 


15* 
15* ' 


"Total responding 


22 • 


* 25 


29 


21 

« 


8 


109. 


* 




il 


b 


• 






Chi square^" 34.3?, p * .10 
(4 group 50 responses omitted from above) 

„ Source: First Follow up • « 

• * 
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tion: "What- do you do with the printouts of^material from EIES?" 
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LIKED AND DISLIKED FEATURES OF THE SYSTEM 

A ' 

V 

The post-use'' questionnaire included* several 6pen-ended questions on 

* 

those aspects. of the system, that are considered to be most valuable 
.and useful vs. those most useless, distracting, or in need of 
Improvement. 

t " • " 

( BIES Favorites * 

) ... ' • * 

Table 5-12 shows the complete, list of ? answers evoked by ^ the 
open-ended question on the '-"most valuable features of -EIES." There 
was np obvious pattern of variation by group, so the answers have 
been rearranged into- rough categories. 

Note that many* members' do not mention specific features at all, as 
was anticipated, but, ratter, general fiaracterlstlcs and advantages 
qf. the mediumi such as the fact that users' " self-orgaMze" 
information ant that the user' experiences the intellecTOl 
stimulation Cof 'a ftflde range of contacts. Among, those who name 
specific features as'the most' valuable aspect of EIES* messages and 
conferences are most frequently' singled out, but' text editing and 
Joint notebooks are" also frequent nominees for "Best Feature." In 
addition, many relatively "mlnor^ aspects of the design are singled 
^out as somebody's very favorite feature, such as "+link" (real-time 
interchange of single lines of text), the "Paper fair,", the multiple 
interfaces, the directory, and even the short but friendly "Welcome"" 
that is the' way the EIES computer responds when first dialed. This 
diversity underscores the conclusion reached in the analysis of "The 
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Evolution of UsAf Behavior" that ..there l&>' no single design based on a 
small number of features that will satisfy the experienced, 
sophisticated user. Users begin to be gourmets, appreciating the 
subtleties of the choices and . variations that can be selected^ from' 
according to preference. 

. / 

A third group of responses focuses neither on general 'medium 
characteristics nor on specific features but, rather, pn specific 
kinds of benefits derived from using EI5S. ' This includes decreased 
need to travel, the opportunity to interact with well-known scholars 
(-the graduate student whb* wrote, this noted that such .collegia! 
cgntact with well-known 5 scholars at other - institutions would not 
otherwise have been possible)," and the ability to obtain such things 
as annotated bibliographies of' Recently published - material, 
contributed 'by the members of a group,* • 1 





.TABL£'5-12 

A LIST OF THE MOST VALUABLE FEATURES OF EIES 
(PARAPHRASES OF RESPONSES TO AN OPEN-ENDED QUERY : 
POST-USE QUESTIONNAIRE) 



1. I especially liked the Immediacy of communication and the 
diversity of • discussions In which I could participate actively or 
passively. It was- fun and intellectually stimulating to be part of 
EIES. * 

2. The asynchronous mode of ' communication l,s the most valuable 
feature; It allows both for delayed Responses and for the delivery of 
messages whenever' and as soon as the addressee returns to his/her* 
terminal. • ; 

J * 

3. Group conferences: The sharing of ideas is valuable. 

4. There are many levels of interface. 

m 

5. EIES is really designed for humans! One does feel free on EIES, 
pot; constrained by\ the computer. It allows the user to utilize 

* "natural" information processes and strategies. Getting information 
from people is pleasant and efficient. Information is not 
preorganized, like in data., bases; it is f, s£if-organized" by the 
users. J 

6., The directory and, the search/retrieval processes are, in general, 
quite informative and easy to use. (And the "Welcome" header for new 
members is great ^s sf first introduction to the people ^ using the 
systenj.)^ . 

7. " I feel no pressure to say anything in conferences. I've ^earned * 
more by listening mofre. m # 

/\ ' * * 

8. Getting annotated reading suggestions is a great learning tool. - 

9. The speed of communication is* a big plus. 

10. -The command that types out all the messages that you haven f t 
seen yet is a great convenience. * (Note: This user had Yiot been 
active for some time. - I think that +GWCfl, get waiting conference * J 

» items, is what was meant.) 

11* The communication involved makes on6 able to keep aware of what 
• some other, people* even though far away in some cases, are doing or 
thinking* '» * * 

// 

12. One can interact, in general, without the customary hindrances 
or inhibitions. j . 
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Table 5-12, Cont. 

13. Thfere is availability of the entire history of a conference. 
Messages can also, be private, and the personal -message exchange is 
very useful/ , , * 

14. There is also the ability to send group messages. 

15* The search for items WOULD be very useful if it w^fked. 

16. The focus is academic, -yet diverse. , ^ 

17. There are also devices to send instantaneous private messages 
and to participate simultaneously in group conferences. - 

19; Orfe can^ quickly tap both special and varied information. 

19. One gets'a real feeling of living in a- network society. > 



20. / 'Then 
lir^Ctly. 



20. y There "are many time-saving system * commands to do things 
dii 

21. The idea 'and potential for research is fantastic. The budget 
constraints caused problems, though. 

4 

22. There are many editing features built intrinsically " into' the 
dystem'. / f 

23+ • \ was able to interact with some well-known scholar^ (arjd with 
the advantage of instant interchange!) 

24. One can update listings of professional activities anytime. 

25. I especially liked the new "paper section/ 1 cl017. 

26. It> l s like having a post box (messages). 

27. Ther*e ar£ a large number of interesting and active people; there 
is always mail or a new conference item of some interest. 

28. 'It' is very easy to sign off. 

29. There is the ability to Beach anyone on the entire SlES system. 

30. , ^Th$ system facilitates shading in the construction of 
bibliographies. 
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Table 5-12, Cont.* 

31. /There Is receiver-sixecif led formatting 'of text (acco{ 
receiver's terminal specifications). 

32*. 'There Is 3roUp 45 (Vocational Rehabilitation of the Disabled)! 

» \ ! ■ 

33. Ti)ere/are joint notebooks for coauthoring and coediting. 

- ■ f 

34. The Chinese menu is useful at*- the ^beginning. 

35. _.0ne can conduct and monitor an evaluation of a, technical aid 
with multiple groups. J * 

36. One is left with hard-popy messages that can be -stored for later 
use. 



37. OnS has quicker, more universal correspondence capabilities than 
one has with the paper mail or over the phone;. one can even tap many 
minds at 6ne time. 

38. The- immediacy of communication, combined with selected 
working-group interaction* and the tfaceability of 

t idea-development-discussion-revision, etc., is unique. I have also 
learned to type from using the terminal. 

f 

39. There is rto n*^d to travel. I also got to interact with some 
new people aqd to/waxch other new people Ir&^vhct* 
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Complaints about' Specific Peatures 

< The opposite of the 1 "Most Valuable* Feature" awards is brought to 
light when one inquires as to those aspects of the system *, that are 
felt to be 'most useless or distracting; they are shown in Table 5-13, - 
arranged, by specially group. Wore %han one comment under the same 
number means that it came from thaJsame respondent. Note that the 
length of the complaint lists and the specifio nature of the 
suggested x improvements vary markedly by group. Groups 30 and 45 have 
very few members who list anything as useless orv distracting. 'On the 
other hand, Groups 40 and 54 (especially relative to the small number 
of post-use returns from group 54) have many .nominees for "Worst 
Feature. n 

* * 0 » 

Group 35 has. severaql complaints about t>he quality of -the content of 

the communications contributed by its own members ("junk messages ," 

"off-the-walO^comments," "making cute remarks").' Similarly, group 40 

is marked by the number of complairfts about group messages that are 

voluminous, unnecessary, or of little general interest. This group 

had the l'arg^est number of "Season's Greetings" consisting of graphics^ 

$tnd text exchanged as group messages. Apparently, some of the 
/ * * 

/members' definitely did not appreciate this particular form, of 

/ ■ e ■ 

l electronic art used as 'social amenity, particularly if they were off 
line during the holiday season, and. had to sit through a dozen or so 
Christmas trees' printing out t in 1 February. As they themselves 
suggest, one solution would be tnat group messages should have a 
self-d«estruct ,N iate. That is, when they are entered, the sender j 
'•should be asked the last date on wtfich the message should be 
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- delivered, since most gopup messages refer to subjects of interest 

* 9 

for only a limi.ted time. 

t 

• Subsequently, features were developed on EIES to allow users to send 
* a short group message that contains ' embedded within it* a much 
lengthier' discussion- for those who are interested in reading it. 
'(There are actually two methods for doing this J one suited for one 
page of material/ the* other, for making many ^pag^es available on, 
request). Currently*, observation- shdw^many fewer group messages 
Being sent and the frequent use of the mechanism of making a- short 
' announcement followed by material that does not print unless selected 
' ! by the recipient. * " • 



Group 54 ls\he only group with complaints, about ,the basic system 
' design. 1 Part of the explanation is probably that many of these group 
-members were used' to working on very sophisticated, * high-baud-rate 
. systems at their own universities. Th^y should probably not have 
used EIE5 at a low baud rate but, rather, should have used micros as 
• terminal**© that they could edit with a familiar editor and scan 
'material at the high baud rate to which' they were accustomed. 



However, there is probably also the effect of an insufficient level' 
and frequency of i use to maintain facility with the system. As Bait • 
'(1979, in Uhlig, Parber, and Bair) points' out, unless people use a 
system It least every few days they keep forgetting what they 
learned, and the system always seems difficult and arbitrary. . Group 
54 never got any successful conference or activity going on Line, 
with the exception jbf a period after Three Mile Island when the" 
nuclear accident laired in . their group conference considerable 

J. 
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• Suggested Improvements * 

V 

/ 

Improved documentation, including lntecactive lesions on line, is one 
of 'the most frequently mentioned areas for improvement" in the EIES 
system. Among 'the other frequently requested improvements are better 
.graphics and better text) editing. Substantial improvements in this 
area wjould require improvements in the^uallty of the terminals used 
by members: t;hat is, as long as the standard or most usual- terminal 
.is a 30c. p. s. portable printing dots on scrolls of narrow paper; on* 
•is not going- to be able to .use sophisticated graphics or editing 

» 

routines. Mathematical symbols, as also superscrlptlng and 
r subscripting, -are also terminal dependent. 

Many other Suggested improvements show considerable understanding and 

insight by /the users as to what can be put into the central system. 

A. frequent category of suggested improvement is lri the "information 

overload" area, from general requests for faster and easier ways to 

■skip the printing of uninteresting-looking portions of conferences or 

documents to specific ideas for how to do this.- For example, one 

member suggests the addition pf the choice "T" (for titles only) • to 

the -"accept ^messages" question, which now allows only the options of 

"yes," > n no, n or the first N messages. ^ 

t 

t 

Another sugges^on that falls^ln this area is the provision of- a 
' high-speed printing service so that large amounts of wait^^ items 
could be printed at the central facility and mailed, rather than 
- printed locally at 30 c.p.s. . . * Such a high-speed printer had been 
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requested in, the project budget 1 but was eliminated by NSF as an 

unnecessary luxury! At the end of the operational trials, as a result 

\ , "I 
of such requests, a high-speed printer was purchased from other 

funds . 



Another frequently mentioned area for improvement is in\ response 
time, particularly for special routines C not -written in hara code). 
This is dependent on acquisition- of a newer, larger machine. 



r ' ^ 
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Table 5-13 

Useless/Distracting, or Out-of-Place EIES Features 

* 

Group ^0 , 

1. Bugs and delays— which are inevitable in the developmental phases 
of any system 

2. Frivolous stuff — games' • 

3. ' All'' the preliminary garbafe before I can get. Into th| scratchpad 

4. More than one line, and faster response, for "+sen^ 

Group 35 „ / 

5. The menu '(but* I assume there's a way to short c/rcUit this) 

6 Terminal errors In the midst of long printouts make it virtually 

material. 

7 Anv Instructions or printed diagrams should distinguish'between 
whaJ^is capable SSon the^system) and what isn't., .Too many hour are 
spent trying to figure out how to work something that isn t tnere. 

3. Junk messages f £ 4 

9. Off-the-wall comments in conferences 

10. Pen names j ■ 

11. Making cute remarks apl^funny games \ 

12. The- editing is Junk. • * 

13. Forced reception— slow response 

Group 40 \ 

14. The introduction is too long. ^ 

Stuff at end-of message— I Would like to be able to shortcut it. . 

List of names in conference shlild be a conference choice; yiot a step 
by its-elf. I rarely use it. ' x ■ % 

15. System commands > , • ! 
People use group 'messages' when they should send private messages to 
thetl* who fhe interested. /Not the system's fault, I suppose^) 
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Table 5-13, Cortt\ 

16. There should be a, maximum time for -the life of each undelivered 

message. It should self-destruct after a while. 

* ♦ 

17. Compulsory^ reception of messages, group messages in particular, 
is a drawback. \ \ " ' ' ' 

The same go^s for conference comments. \\ should be made easier to 
• skip a CC and go on to the next. This is possible by SCM but very 

awkward -and -so In ,faa£_n£t a_j>racttfcal option. «. 4 

\ --- - — __________ ^ j^*^ 

18. The Newsletter is too specialized and too frequently adverti&ecf. - 

19. ''Chit-chat 

20. Gfoup messages * % 

21. The volume of text to be read. Editing. • - 

22. Too much paper pours put. IfW r can afford it I will ; get a CRT and^ 

print only selectively. ■ \ 

« 

23» Certain undesired group messages, large "Merry Christmas" 
greetings, etc. '(Junk ma!il) 

24. 'Sometimee the extraneous group messages are a pain. 

25. Too many group messages ? 

26. Have to wade through too much useless material 

27. Group messages that are Apt really of general interest (e.g., 
"Merry Christmas" notes) 

28. I/O is slpw and -difficult (for me). 

29* Difficulty of keeping track of last item I saw In a conference 

Not being able to respond 'to messages or conference items right , away, 
just after tHey are displayed, without some commands 

Burden on memory of too many cues, e.g., 2, 4, cnc, cnm*, etc. * 

Slowness of system, especially in composing messages 

Lots 'of the messages and comments are Insignificant. 
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. Table 5-13, cont. 

Group 45 # A . v . 

►30. Allowing .All users access to all other users — i.e., it would be 
nice to be able to disallow message sending at jwill. 



» 



Overall, ttjr'get many ''complaints" that the "system .is too difficult to 
7Sfc**a r -4^r'oetter ".manual," in particular, , is needed) and that *lt can be 

very slow. 1 > 9 ' 

1 » * , 

/31. Delay at CI\ * ' . 

Group 5^ • * 

32. The terminal, phone patch, etc. really gets in tke way with this 
system. And ^the systejn architecture, formats, and input-output 
routines are not as easy to learn and ' remember as they could be. 
When the basic system structure is completed and it all works OK 
y-ou'll have to go Bteek and work on streanrlining all these things. 

I want to empH&size how long' and awkward the gy$£em learning process 
seems* to ^noncomputer users J >Also, . in our fast-paced worl<3, the 
learned operations and procedures .seem arbitrary ^.nd are 
easily-quickly forgotten fndm one week ,to the next. : Revision of the 
Red Book to streamline and provide ' ALWAYS handjr, ready reference 
would be-worthwile. . ' 

33. Graphics. Long lags in time sharing. Poo£ editing. 

34. I am afraid. I found EIES to be awkward and backward. This is in 
comparison together systems of teleconferencing (e.g., FORUM) I have 
used. ~ 

Specifically v 1 f 

A. EIES is very slow. * 

b. Too many arbitrary symbols to know 

c. Impenetrable interface to other systems 

d. Inability to accept files, in computer compatible form (tapes, 
etc.) 



e. Poor documentation 



\ 



s .Table 5-11 

. , < *•* 

' * SUGGESTED. IMPROVEMENTS 

P t ost-Uae Responses to ,the Qmestion, 
"What general improvements/new features/changes would you like- 

to suggest for EIES?" 

0 

l/ Easier ways to skip, over unwanted text or uninteresting portions 

of text ' 

• * 

2. Graphics, very tough now * * 
3* Mor« reliable system 

4. Paster, ^.ess cumbersome text editing 

5. Better/easier ways to organize,, string together, or hare 
group-related comments 

6. Improved graphics capabilities . K 

7. Sudfeary message header list, (i.e., when one is asked i£ or& wishes 
to accept JaT^at/ of waiting messages, one could answer ,f T" for titles 
only, or '"H") ^ ** ' 

8. ^ Interactive tutorial for learning advanced- features* + ^ . 

Would have helped a .lot if interactive ^programs, e.g., simulated 
interviews, had run more quickly. 

9. High-speed printing, gashed item .searches . - 

10. Ability to retrieve data from other facilities' 7 

"11. Easier ways to* skip 'over unwanted text or uninteresting portions 
of text 

;Bette£ graphics and mathematical symbols 

♦vacation. This 'would * allow you to define a message, explaining 
that you were on vacation (*or out of town on business, or whatever), 
Including dates, etc. Anytime * someone sent you a mess age, you r 
explanation would be 'sent to them, automatically, then Us^y^dkn^w. vttiy- 4 -*^^ 
you weren't answering urgent communications! , 

, «*« ■ 

13. *&lmplify to bane* essentials;- replace with long-distance 
telephones > s / % . ^ 

' # 14. Either /increased computational ability- or 'increased data^ access 
to' other systems • 

* i m , 

v* 15. Some* data analytic capability > - 
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Table 5-14,- cont. > * 

16. Fewer limitations on. length of messages, better way 'of handling 
(i.e., faster) line overflow 

17* Math symbols* 

18, Multiple reference numbers referring to past messages. Enter 
conferences- in the directory, Requirfe announcements when conferences 
are opened* 

* % 

19, Superscripting, subscripting. Better editing.^ 

20, Text editing and message-composing treatments are so inadequate 
and difficult to learn. Documentation inadequate. Need "primer," 
better reference manual. ^ 

Provide option for large volume of output' to be mailed, 

21, Bibliographic reference" files with key words, etc, 

22, I would like to see a matrix of who talks to whom available each 
month, 'like timestat. 

I would like -to \be able to order a 'set of printouts and have them 
mailed to me, 

'23, An MIT professor recently visited and noted that EIES cannot be 
taken 'seriously until it gets a better editor. The designer- should 
look at the DEC 20, RT-11, ' or other good editors Foi< ideas. 
Specifically, the automatic renumbering' of lines needs to be done * 
away with, 

24, Longer hours of the day 

25 • improve the facility .to search messages, or CC f s by author, 
keyword, or date, 

QET OFFLINE PRINTING ! so I could get'a'print of selected cc's, etc. 
Forcing all information through the 30 cps bottleneck is currently -an 
irritation and "an inhibitor of participation. 

Sometimes response times on EIES seem quite slow*. I would rather be 
told "EIES is full, try later" and then get fast service/ when* I DID 
get on than be a participant in a sluggish System slowed down by too 
, many users. - 0 ' 

26. I would like to *ee -tutorials, workshops, or lectures on EIES. 

27. Some kind of quality monitoring* ' . 

28. A way of accessing text via abstracts 

29'. Some sort o£ jLnformation filter — e.g., more ' summaries jj^ith 
details available' if wished 
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Other Evaluative Comments 

For, the sake of < completeness , responses to * the final open-ended 
question included In the post-use questionnaire are shown_ in? Table 
5-15. These are a "mixed bag, though mostly on the positive sid«.. 

There are some -qualitative measures of subjective satisfaction, that , 
with hindsight, it would have been good to research systematically . 
One is that several ^members of" the system were so enthusiastic about 
it th^t they wrote professional pap|rs on it^and otherwise efigaiged 'in 
proselytizing efforts. About ^a half dozen of these have been 
received; if a question had been included on the post-use instrument,, 

r 

' more might have been turned up. 
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' * Table 5-15 




Other- Reactions to EIES' ; 



(Question: "Any other comments on. ttfie EIES system or Its Impacts?") 

1. I was terminatedjust at the time when I was beginning to realize 
the potential benefits of participation on the*system. A year isn't 
time enough if you are holding down k 'fulltime tfob and all sorts of 
other committments— unless you have a nj^s block of released time. 
Thus, I've ended my us6 with an intense feeRng of f rustration-^but^ 
I'm not sorry I -participated, nevertheless. • - 

2. klES attracts its own "nuts;" also addictive like drugs* 

ft •m ' 

3» If EIES ever" gets another grant that would ailow me to .rejoin it; 
I would be_very* interested.^ I think it is a marvelous, aid to 
stimulate thinking, compare theoretical conclusions, etc. 

4. The system was not as useful, for me, as i#*n\ight have been solely 
as a result of the participants .and the types of things communicated. 
There seems to be a natural, tendency for the discussion to degenerate 
to trivially abstract issues so that the best persons in network 
analysis gradually stopped signing on. If there is any one .main 
problem, it. is the lack of social constraint face-to-face interaction 
puts on the exchange of trivial items in professional discussions. 

5. Pace-to-face meetings — whicfi* probably would not have occurred 
without EIES— have helped to generate a sense of belonging to a 
frsub Community. 

6. A more coherent research specialty group would benefit more from 
EIES* It would also be' useful if> EIES were available' at every 
university foP conferences, -to .be shared by, researchers in many 
disciplines* 

7. I respond to EIES requests in days,, whereas; I respond to mail 
requests in months. (Why?)' < * 

I write a lot to EIES, but almost never write letters. 

8. I LIKE EIES despite my fcrouchiness, about the apparent 
unwillingness to provide offliife printing. It is imperfect but a 
friendly first attempt at a usable computer conference center. The 
consultants- are very helpful. I wish I had time to explore jnore of 
the conferences on the system, for I know there is ^a lot going on. 
that I have only glimpsed. . 

9. I think that it is important to hate high quality ^scientifically 
speaking) participants. Most of what I -saw was worse than second 
rate* * 
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Table 5-15, cont. 



* 



9l0. I- enjoy it after overcoming several difficulties. It is DIFFERENT 
and therefore not so'easily comparable .with traditional modes of 
communication. ' It has ITS OWN style and way of exchanging 

. Information that is likely- to grow'rather than replace. others. 

I will miss it when the pro ject. is terminated and I might not be able 
to afford the costs of participation on my own. 

11 % EIES is like marriage — can.'t stand it but need it.' , 

12. I -wi'sh EIES were more "secyice oriented," with cheap, rapid 
distribution of materials by mail -(printouts 'as microflce perhaps?) 

. Clones updating each other at acceptable transmission rates wouLd be 
idea^ Biggest threat to its future will' be legal, political issffps.' 

13. I I think EIES is great. I would l^il^Xto participate more, and I 
feel* it hasfa great future v , v / 

14. Given funds, the most important decision an individuar^ has to , , 
'make, is how best to use his/her TIME. I have f<5und that \he EIES 
experience is an extremely valuable learning exercise, but^W\en I ^ 
have much to learn. ^ j 

4 15. EIES has*forced me, to t>ruly appreciate changing \ technological 
shifts /and how to "cope" in a positive manner. I 

16/ The review of communications xeroxed and distributed by Umpleby 
(^roup leader) was very useful. That kind of review should probably 
be dorte quarterly so that those not using the system much could catch 
up and not feel quite sp reluctant ta reenter' the communications. 

17. In general, I have been very pleased with* EIES as a communication 
medium. *I have been Unable, due to the difficulties of gaining 
access to a terminal on a regular basis, to spend as much time on the 
system as il would have liked* to. In my time on EIES, I have bfeen 

. pleased with some of the people on the system but hive generally 
found most of the comments and interactions to be worthy of little 
more than passing interests What has jpeen very profitable has iieen 
the use of private conferences for getting something practical donA 

18. Keep EIES whatever the cost! 

19. Once you have established a link, attention for a problem is very - 
good, but the problem is to g«t through, to get attention for a ^ 
specific question. A lot of things get drowned in the flood of 

.information. 

. 20. I am proud to have been a part of it. 

* * * 
•21. We need a full-time group member on the system to set up 
structures . that other members need but, lack the time to initiate. 
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, Table 5-15, cont v 

22 • I have truly enjoyed arid benefited from It but have not gotten to 
u-se It In the past several' months because of travel* time to Newark, 
In order to be of maximum benefit,, it' must be on-site. To travel a 
distance, even 10 minutes, reduces one f s abilities^ to utilise t^ 
system. 

23. A problem with its use for some purposes is the limited number of 
participants who are on line. * 

24. The electronic journal experiment has been very disappointing in 
practice. The quality of, EIES has, in this case, turned off all 
users. ' Our evaluation (and other peoples' too) in a "volunteer use" 
situation is not a good test. If my institution or my profession 
made it de rigueur to use. EIES, I would use it and enjoy it. The 
trouble is that' the affairs of my and others' professional lives are 
conducted via another medium. Thus, the EIES experience is not 
"real," and there is little motivation for people to utilize- it. 

r 

25. This system nee<ts to be tailored to particular kinds' of 
interactions on specific kinds of topics. The payoffs may come 
through exchanges of 'factual information on how to do an experiment, 
for instance. We know what kinds of information are" exchanged by 
phone, letter, presentation. Journal article, and book, for instance. 
However, we have nb expectations involving this mode, and-, to be 
effective, such 'expectations should be formally declared at the 
outset, kind of like an up-front contract. 
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• OVERALL REACTIONS TO EIES 

* f 

Oije measure of, satisfaction with the EIES experience lies in the 
feeling tjy participants jfchat they received benefits at least equal to 
the effort expended. The majority of participants, ^s shown in Table 
5-16, do feel that they received a^s mixah or more , than they 
\ contributed to \ thel y group(s). The most active participants (100 or 
more hours «on line) are^cost likely to perceive' a balance between 
their, contribution^ aryi^the amount and quality of information 
received as a yresult of contributions by others. 0 Somewhat 
surprisingly, participants at the' lowest levels of activity, who in 
fact are most likely to receive much more than they type into the 
system, do not always ^perceive this to bq the *case, and, at 

intermediate levels of use (50-99 hours on line by the end of * the 

** * 

trials), there is the greatest probability that participants will 
feel that they are contributing more than they are receiving in 
aeturn. 



\ 



Responsi 



veness to Messages 



Another overall ' measure of ^atisfa'ction with thij form of information 
exchange results from the p'ercep^on -of the responsiveness of others 
to *he messages they receive. We, once again see a» somewhat 
curvilinear* relationshitL to amount of time on line, with 
intermediate-level users forming the < highest proportion those 
'feeling that people are less responsive to EIES messages than they 
are, to mail or to phone calls. None of the most active participants 

feel that others are less responsive to electronic messages than to 

, * i 

j 

4 
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other forms of communication. This is an interesting perception, 
since many of the persons to whom they send messages must be on' line 
much less fbequfently than they themselves are, 

I Experiences while Communicating over EIES 



Table 5-17 .shows the frequency, with vrhich users report various 
experiences or feelings while using EIES. For example, the frequency 

* I 

with which one feels distracted by the mechanics of the system 
intruding upon the. flow is related to group Membership. There* is a 
tendency for this feeling to decrease with more* time on line, but the 
relationship iS not statistically significant (gamma * .17, p = .24). 

For feeling "overloaded with information", "sometimes" is the- modal 
answer. The frequency of feeling an overload appears to peak, in the 
middle ranges of use; 315 of 4 thos^ who had logged 20-^^ hours on line 
report "almost always" experiencing information overload, whereas all 
of those who have logged 100 hours or more report 'the overload 

* experience to occur only "sometimes." rfeing. able to express your^ 
views is generally reported to occur "almost always;" among tjhose 
with 50 hours or more on line, the responses are all in tjfie -"always" 

*)or "almost aiways" category (the correlation with hours on line as 
measured by gamma i? -.20, significant at the .05* level). 

S 

In terms of feeling "constrained in the ty^es of contributions you 
could make," "sometimes" is' also the m.odal answer. Finally, being 
"able to get an impression ' of personal contact with *other. 
participants" tends to occur "sometimes" for tfrose w^th less than 50 
hours on line and "always" or "almost always" for those with more 



212 



t * 

than * 50 *h ours on line (correlation with hours 6n line, a3 measured by 
gamma, m is -.47, sigt^ificartt at the .02 level). 
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Tfabl-e 5-16 > x N . 

> 

Some Impressions of EIES, by Time" on Line. 

• <» 

A. Balance between Contributions Ma&e and Information Received 

•* 

Contribu- C'ohtri- Equal Received Received 



buted 



More 



Sig. More 



1-19 
20-49 

50-99 
100 + 

• ■* 

All 

/ 



More 


More' 










- 4% • 


7 


37 


37 


- 15 


27 


°* 6% 


•9 


36 


21 


. 27 


33 


12% 


28. 


'28 


12 T ' 


20 


25 

t 


' 0 


JLO . 


• 58 

• 


* 


jU 


- 19 


6% 

• 


14 


38 Y v 


22 . 


/ 20 


104 



Source: Post-Use Questionnaire 
Chi 'square * 16.5, P *M7, gamma .14 



Question: Comparing my contributions or effort put into EIES with the 
amount of information received, I feel that I have: v contributed 
significantly more than I have received, contributed more than I have 
received, contributed as much* as I have received (equfcl), received 
more, received significantly more than I have contributed. 



B. Relative Responsiveness to EIES Messuages 

Total 



More 
Responsive 



Less 
Responsive 



NO . 
Diff ertfnde 



<5 hours 
5-9 hours 



40} 


' *0l ' 


60% 


iob% 


5 




' 38% 


36%* 


10'0% 


42 


49% 


23% 


2$% 


10"0% 


35 


33% 


17% 


50% 


100% • 


if 

12 


72* 

* 


.0% 


28% 


100% 


7 




Chi square - 25.894, 

f 


p » .0967 . 

s 







100+ hours 



Source: Follow-Up Questionnaire 
Question: When you send a message over EIES- rather than writing or 
telephoning; would you say that recipients are generally: More 
responsive to an EIES message? Less responsive? Equally responsive 
(no difference)? 
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Table 5-17 

' Experiences while Communicating via EJES ; 

\ > 

✓ 

(Question: Thinking back oyer your experiences with the system, how 
frequently have you felt...) 

Always Almost Some- Almost Never Mean 
Always times Never 



/ 



(1) 

5% 



4JS 

4SS 
24% 
' 8* 



Distracted by 
the mechanics 
of the system 

Constrained-1 
the types of 
contributions 
you could make 

Overloaded with 
Information 

> . Able to express 
your views 

* Able to get an 
impression of 
personal 
' contact with 
other 

participants 



Source: EIES Pollow-Up Questionnaire, N » 110 



(2) 
16* 



17* 

18* 
'47* 
35* 



(3) 
49* 



44* 



V) 
23* 



55* 
24* 
46* 



23^ 

16* 

5'* l 
6* 



(5) 
7* 



8* 

6* 
0 
5* 



3.1 



3.2 



3-0 



2.1 
2.6 



ERIC 



2'2'J 



SUBJECTIVE SATISFACTION SCALES 

**'.-• 

Ovejjaii^ratings of EXES as a "communication-information system tend to 
% be fairly, positive ,>, but not "perfect," in terms of users' subjective 
responses to aL riumber/ of . scales designed to measure satisfaction. 
Subjective reaetiqns are, needless to say, highly correlated' with 
tot^l amount of use of the system. \ • 

Table 5-18 shows that, users tend to rate the system as good overall 
by the time of the three-month follow-up .and also as stimulating 
rather than boring, productive, fun, friendly, and easy to use*. 
There are three dimensions on which a quarter to a third of the 
respondents give , a negative rating: that the system seems to be 
frustrating at times, time wasting, and intrusive. - 

These subjective satisfaction scales are the most general assessments 
of EIES^yJh&f^ we have. They* "will be used as the basis for a more 
details analysis of subjective satisfaction factors and their 
determinants, which will be presented* at the end of the chapter. 



The DACOM scales, designed originally by the Communications Studies 
proup in Qreat Britain, have been'used to measure users - perceptions 
of a '.vaidety of systems and media. Chapter Eight will present some 
of the comparative data available. For this study, scales were 
administered both at follow-up and at post-upe. There were very high 
correlations between the measures at the two points in ' time. The 
post-use data were ?ehosen for Table 5-19, since the follow-up 
measure? have been presented in interim reports. EIES is seen ^as 
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most satisfactory for emotionally neutral" task-oriented functions: 
'giving or ' receiving information, exchanging opinions,, generating 

ideas, giving or receiving orders. It is' also seen as, satisfactory 
.by most people for social-emotional tasks such as getting to know 

someone and expressing positive' and -negative emotions. It is 

perceived as least satisfactory for functions related to conflict and 
/ rfegotiation: problem solving, bargaining, persuasion, resolving 

disagreements. For these - last tasks, the ratings cluster in the 

neutrals (3-50 range rather . than in the positive (l r 3) range 

characteristic for other functions. 

Two groups using the system (JEDEC and a medical standards group 

called MRP IT) reported that a 'characteristic mode of communication 

mix was to use the system for routine communication and to resort to 

other modes, such as face-to-face meetings or the telephone, when 

^conflict arose. Whether special structures can be incorporated into 
k 

computerized conferencing systems to support conflict resolution is a 
research question that is now being pursued. Without such special 
structures, it is evident that user groups find the medium lacking 
for conflict resolution tasks. 
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Table 5-i8 

Overall. Reactions to the EIES Mode of Communication ^ 



Extreme- 
bad 

11% 



78% 30? * 3% 



: 1 , : 2 : • 3' : 4 
Boring lating Jd * 

' 20% 37% 27% f , 7% 



: 1 : 2 
Productive 

6% -22% 36? 



3 



: 1 : 2 
Unpleasant fun 

15% 36% 



22% 



: 4 
16% 
: • 4 
14% 



\ 



: 1 : 2 
Time saving 
' 10% 



3' 



15% • 19% 24% . 

2 : f : * 4 



f rus- 
ifcg 



8% 



11% 22% 



Friendly 



4'2% 



28% 

4 



22% 

3 : -4 
14% 



Difficult . 

16% 28% 22% 



: i : 6 ' : 7 : Mean 
Extremely ly good 

10% .'.4% 0 2.8 

. 5 : ♦ 6 :• 7 : Stimu- 

8% 1% ' 0 2.5 

: 5 * - 6 : . 7 : 
. Unproductive 

14% 4% " 3% 3.3 



5- 



12% 



22% 



18% 13% 



: 1 : 2 : 3 : 4 
.demand- 
ing or intrusive 

14% 16% 20% -22% • 24% 

- / 

Source: Long follow-ups, N - 111/ * 



6% 



7 ' : Great 
work 
2.7 - 



7 : Time . 
wasting 
4% 



: 6 : 7 : 
Frustrat- trating 



23% -9% 



4% 



3.7 
Not 

3.9 



5:6:7: 
Impersonal >* 
6% 3% 0 2.7 



5"\: 6 :' 7 ; Easy 

3% "0 2.9- 

5' : 6 : 7 : Not 
Very demand- ing or 

intrusive 
3%- 1% 3.4 
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- . -fable 5-1-9 
• "CSO' -DACOM" Scale?: ^ 
Extent ta Which EIES Is Satisfactory, for 
Various Communications .Tasks 



Completely 
< Satisfactory • 



Completely - 
"Unsatisfactory s 





1 1 


2 


3 


4 


5 


- 6 


7 


Mlans 


Gamma 


p* 


' Giving orv^ I 
' receiving . 
Information 


25% ^ 


J* 


I 

r: 

0 


♦ 

10 


7 :. 


4 


*0 


2.4 


• 17 


.70* 


Problem Salving 


3* 




19 


■ 28 


23 


7 


4 


• 3-9 


.15 


.22 


Bargaining 


6% ' 


9 


16 


30 


2Q 


a 


9 


- 4.1 


.16 . 


.65 


Generating Ideas 


15% 


30 


'33 


11 


7 


# ' i 


3 


2.8 


•29. 


.46 


Persuasion 




5 


29 


20 


19 


15 


8 


4.2 


-23 


.02 


Resolving 
dlsagr elements 


• 


7 


28 


*3 


16 


14 


7 


4.1 


. 18 


. 11 


Gettlng(to know 
someone \ 




< 

29 




14 


7 • 


7 


4 


' 3-3 


-.21 


.26 


• P4 4 i 

Giving 

receiving orders 


ioz 


34 




22 


/ H 


5 


L. 

* 0 


< 


> 


,.65 


Exchanging 
opinidns 


T 


£42 


20 


6 


5 


1 


2 


2,3 


■ '.18 


.26 


s Expressing 

positive embtLons 


7% 






16 


8- 


4- 


6 


3-3 


..04 


.20 


4 .Expressing 
negative - emotioft? 


■7% 


22 

< 


22 


^2 


16 


'5 


5 


3.5 


.17 


.66 


Sociable v 
. relaxation 


* 2? 


21* 


27 

• 


21 


11- 

• 


7^ 


10 


3-9 

7 




.42 



• Source: Po'st-Use Questionnaire, N » 102 
*p (probability*) is level of significance, based, on Chi' square 
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* Subjective Satisfaction Ratings, and Time on Line 

. The subjective ratings of EIES , do tend^o be positively related to 
accumulated hours on liae at 1 the* time the questions N were answered. 

, However,* mo£t of the relationships are weak statisticafty 
ins^gni^ic^nt. The overall racing of the system (EIES is extremely ^ 

"""^ood-extremely bad) is significantly related to -time on line (Chi 
square * 32". 6-, p - .04, gamma*- -.45). The only other scales showing 
a significant relationship ar$ personal-impersonal (gamma - -.24, p - 
* J^l) and time saving-time* wasting (gamma - -.28, p - .05). 

Group Differences o # 

Most "of . the DACOM scales do not show significant differences 
•associated with thfe -user group. However, some do. Using the system 
* for persuasion * is most highly rated by mehbers of .Group 40 and 
received the most^ unsatisfactory ratings from Group 45 (see Table 
5-20). Resolving disagreements, significant at only tke .09 level,, 
showed a similar pattern. This is to be expected, ^sdrffce they are 
similar functions. 

. / 

gov "getting to know someone," Group - 30 is the most positive, 

.followed by Group 40/ and Group 45 is ,again the most negative. For 

* * / ' 

giving and receiving orders, on the - other hand, Gfoup 45 is more 

v split than the other^.and Group 54 t8 the most decidedly neutral 
(see Table 5-22) \ . 
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'None of ^/the ) other: scale items show differences 'among groups that are 

sign\f icanj/ .at the .TO level or above. The differences that do occur 

/ 

indicate /that the specific experiences of the group do have some 

** * ,. 
eff ect^^Hipon ratings of the degree- to which the^system in the abstract 

is suitable for some coramunicatio^^rposes. 
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Table 5-20 

Satisfaction witfe EIES for Persuasion, \by Group 



m Completely^ . 
f Satisfactory 



Completely 
Unsatisfactory 



Oroup 


l ; 


: i{ 


: 3 : 




• 5 : 


6 : 


7 


: N 


30 




t 7*< 


' 2736 


20* 


7* 


• 33* 


7* 


15 


35 




16% 


28*, 


\12% 


20* ' 


12* 


8* 


25 


i»0 


7*' 


0 


45X - 


h ■ 


21.* 


10* 


7* 


29 




0 


0 


' 5% 


37* 


32? 


10* 

/ 

17* 


16* 


19 


54 

* 


. 0 


0 


17* 


" 67* 


0 


0 


6 


Aii 


3* 


5* 


28* 


21* 


19* . 


15* 


9* 


94 




chi square » 38. 3,,. P ■ .03 
contingency' coefficient ■ .5^/ 
Source: Post-Use v Questi>n4^a/re 
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» ■ . Table 5-21 

Satisfaction with EIES for Getting to Know Someone, by Group 



Completely 
Satisfactory . 



Completely 
Unsatisfactory 



Group: 


1 


2 


: 3. ••: 


4 : 


5 


: 6 : 


7 : 


N 

• 


30 


0 


69* 


19* 


0 . 


12* 


0 


0 


16 


,35 


0 


23* 


42* 


15* 


4* 


12* 


4*- 


26 


'40 


10* 


J 2b% 


41* 


10* 


7* 


0 


' 3* " 


—29 


45 


0 


16* 


32* 


21* 


5* 


16* 


■ lot 


19 


54 . 


o- 


11% 


33* ' 


33* ■ 


17* 


0 


0 


6 


All ' 


3* 


30* 


' 35* 


14* 


'7* 


*6* 


4* 


96 








chl square 


« 35.7, 


P " 


.06 ^ 






• 






contingency 


coefficient 


- -.52 







Source: Post-Use Questionnaire 



ERIC 



223 



237 



7^ 

. Table 5-22 ' • 

i 

Satisfaction with EIES for Giving- and Receiving Orders, by Group 



Completely 
Satisfactory 



Completely 
Unsatisfactory 



Group 


• 


1 


: 2 


: • 3 : 


4 




: 6 


: -7 


: N 

r 


30 




• 

0 


46* 


8* 


23% 


23* 


0 


0 


13 


35 




13% 


o of 

39% 


0 


355 








O 


ft r\ 
40 




us 


37* 


nil 


7* 


' 4* 


*♦ I* ^ 


11% 


0 7 

C / 


45 




lit 


32* 


21% 


16* 


0 


10? 


10% 


19 


5^ 




0 


0 


33% 


67* 


0 


0 


0 


6 


All 






35* 


15% 


23* 


6* 


4* 


7% _ 




i. 




\ 




chl square 


- 35- 


8, P » 


.06 














contingency 


coefficient 


- .54 












* 


Source: Post 


.-Use 


Questionnaire 
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ATTITUDES OP OTHER HOUSEHOLD MEMBERS , 

-J 

. ' ' • • / - • 

A communications system like EIES potentially generates reactions not 
Just from direct users. Others observe interaction with tlfe system 
and form opinions* about whether this on line activity adds to, 
detracts from, or is* neutral in terms of its effects on their 
(off-line) relationship. The most- important^ef / the potential groups 
which there may be a seconda > r>^4«partit^^ the family, particularly 
if the network member uses a terminal at home, and particularly if he 
or she ties up the only phone line. * 

< 

5Uny EIES users do not take their terminal home or- talk to their 
families about their work; their families or living^ partners, are 
oblivious- to It. Por -those who do take It home, reported reactions 
vary from great curiosity and enthusiasm to hostility and resentment 
(see Table 5-23). 'Reactions of interest, curiosity, and support are 
much more frequently reported than are ^negative reactions. 

A very lively debate on the impacts of EIES on family life occured In 
the public conference on "Impacts". Opinions ranged from the point 
of view tiyft^DC will worsen the detrimental strain that TV and other 
relatively modern technical developments have put on family trends" to 
the assertion that it can strenthen the family by, for instance, 
allowing spouses separated by travel to remain in contact or 
permitting parents to work at home rather than leaving their children 
to ^todfi office. The intensity of many pf the comments on the 
."pros" and/ "cons" of having a terminal in the home indicate that the 
reactions of other household members to CC J9T^ well as those of 
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Table 5-23 

t Responses of Other Family Members or Friends 

"Seems like a fun thing that- I am* doing, but It Is no big deal to 
.them." . ' 

v 

"Curious fascination to Irritation (when I bring the terminal home).* 
' "Huh? they couldn't care less." 
"Very interested." 




f 5rj£ftusiastic, interested, envious in friendly fashion; and they 
learn things from EIES." 

7My wife, is moderately .interested. My'children are enthusiastic,. 1 

"Interested. Look for future developments in this technology. ,f J 

"They dislike * my * keeping the phone busy too" frequently and tog long 
each time." 

"Curious scepticism." 

"My wife like* it a lot. My wife checks the messages and 'talks 1 
with the .systems people." 

"Ki$ play.8 'animal' on visits." ' \ 

"Don't know or care." > , 



"Think -it is . a fun "toy. Are annoyed at tying up the telephone. Are 
interested in messages that they understand." 

"They hardly know." 

» 

"That it's great and should be expanded to all areas of 
communication." t ... 

"Oblivious." 

» 

"Tolerant; not excited at all." 

"Positive." , . 

"Children neutral. Wife negative." 

"I have been forced into mainly working on EIES after 5:00 pm because 
of telephone rates. My occasional latenesses in returning home annoy 
my wife." ' / 
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Table 5-23, cont* ' 



"My 'husband interested and a bit envious. My children are- too 

underst ~ 
matter-of-factly . " ^ 



young to understand ^^iat it is all about, . bu& accept it 
"Indifferent n %. 



"Mildly interested." • 

"They find It terribly exciting; f A giant intellectual, C.B. , ' as one 
of them described it." , 

"Impressed." «- 
"Amusement and amazement." 

C 

"Interested." - 

"They don't care. * My son likes, it when I bring the terminal home (a 
practice I just startecf) . so that he can use computer games on 
another system." 

"Encouraging." 

"They have no attitudes whatsoever toward it." - 

"They know nothing about it. It's my dark secret." 

■ i 

"Yet 'ANOTHER activity' to distract me from family iife\! But generally 
supportive!" 1 

"Respect and admiration." 9 

"Wife enjoys it, finds it interesting and amusing." 

"They are disappointed that, .unlike other computer systems I interact 
with, EIES has no provisions for interstellar combat and similar 
diversions." 

* 

"That damn ^wfiputer." 

"Wife: indifferent. Childreri: sompwh^t curious." 
"Supportive. " 

-"They think it is interesting. A^like a toy.*" 
"Amused*— sometimes annoyed" ^ 
"A distraction, but they accept it as important." 
"Love it." 

"Not involved." J 
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Table 5-23, cont., 

* 

"Intrigued" 
"Enjoyable." 

"My wife is excited about the idea and the' system. " 
"Moderately interested." 

"Very* positive— after we got an >textra telephone line for the 
terminal." ' 

\ "Positive except when 1) paper accumulates throughout the house, or 
2) I become frustrated when system is* slow or I have difficulty 
accomplishing what I Intend to do." 

< "Between EIES and my home computer they sometimes wonder who that 
strange man is irt the study." ' 

"Support^Te, interested, excited." m % 



"My cnlldren are not involved in and not aware^of EIES. My wife 
knows about it and thinks 'it's* great." 

"Wife>*s a user." * , • 

"Wifivhas mild interest when I' take it home." 

"Enthusiastic— amazed." , > 

"I don't use it £t home. If , £ did, it , might compete with family 
activities." . N ' 

"They think ,it is somewhat useful, but, since they are not as 
interested in computers, they are nQt enthusiastic. "J 1 

* * • 

"My wife is excited about ^he idea and the system." 

"Moderately interested." 
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4 . WHAT USERS ARE WILLING TO PAY 

.1 * 

Those of ±\ practical nature often state something to the effect, that 
people f s* attitudes or opinions about a new product or k service ¥i^e 
beside tjje point* . What really, matters is "the bottom line" — will 
enough people pay enough for the service to make it economically 
viable? f 

One- pr obi em w^t^the generalizability of the EIES results is that the, 
scientif leaser-groups were nSt paying for their, use of the system. 
At th&r current time, in fact, ' it is not economically feasible for 
either an independent scientist or strained academic departmental 
budgets to pay for the costs of using this form of communication. 
Even though EIES is both nonprofit and designed to be a /low-cost 
system (using a minicomputer rafher than a large mainframe computer), 
the , costs are* over $100/raonth per member. (1981 costs' are $75 per 
month for system use, plus the cost of connecting tp the system via a 
packet switched network and/or the telephone. During the operational 
trials, TELENET cost $3*75/hour. Rates are now in the $5.00/hour 
range. Assuming 10-20 hours of connect time per month per user, this 
is well over %lb0 a month to use EIES, plus the cost of a terminal). 

S 

Table 5-24 shows the median amounts that EIES members said they would 
be willing to pay for the system, under various conditions. * The 
variables are whether the people are joying out of their own pockets 
or are being funded from some r other source and whether they would be 
continuing as members of 'their operational trials research community 
or would be able to put any group they wanted on line. * We have 
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omitted from the calculation of the means and medians in this table 
respondents who listed "$0" or who said they would pay whatever it 
cost. A 

/ 

One can see* that the amount that the scientists fel^~*they would pay 
out of 'their own pockets for continued membership ^-tri the EIES 
research communities .(a median of $3«.50/hour) would- not even-pay the 
TELENET costs, "let alone pay for system use. However, the amounts 
they would be willing to pay Under oEher conditions are in the realm 
of economic feasibility in terms of supporting system cpsts. 

•This analysis has been borne out in practical terms in that enough 
members were willing' to pay the costs of EIES after the end of the 

'operational trials to make the system self-supporting. However, the 
majority of these- pay-your-own-way users are from industry or 
government and are' not^aying out of their own pockets. We come to 
the conclusion -that, no matter how valuable systems like this might 
be- for scientists, they are not likely to be abie to use such systems 

'unless they are subsidized, as they are for--other research tools they 

\ 

use in their work, from libraries to nuclear reactors. 

t 

» 

MULTI-VARIATE ANALYSIS: FACTOR ANALYSIS AND 

» 

STEPWISE MULTIPLE REGRESSION 

* * 

An attempt at multi-variate analysis was hindered by the number of 
cases available when using many variables from the pre-use and 
follow-up questionnaires: if there is no answer available on one of- 

i 



the '.variables used-, then -the case Is eliminated from the analysis. 
."And . the more -variables one puts intjo the 'multi-varla-te analysis, -the 

"V * ■ - \ . 

worse- It gets. Nevertheless, we were sucessful In determining- some- 

« 

Interesting clustering jzt measures of subjective satisfaction, and - 
some import^njt 4«terminants . 

The Items shown In Table 5-18 were subjected to one of the most 
widely used approaches to factor analysis, the "PA2" approach (See 
Kim, 1970) with VARIMAX rotation. ^ (This is* the "normal" or "default" 
.type of factor analysis in SPSS). The purpose of the factor analysis 
was to see how the various dimensions of subjective satisfaction 
measured empirically cluster together, so thafr'some of them may be 
combined to derive an index of some underlying factor which several 
of the individual questions have in common. The results are 
presented on a graph so that one can actually see how close together 
# the individual questions are when classified by the underlying 
•factors. w ' . > *' 

*• • 

Two underlying factors were Identified. They seem to correspond to 

,1 

"input 'frustration" or difficulty, and "output payoff", or 
v satisfaction with -what one gets out 6f the system. Three questions 
t were right 'in the middle op^hth factors, which makes logical sense, 

because they *Sqrrespond to a kind of balance between input difficulty 

and output payoff. % > 

» « • 
* , * 

The "varlmax rotated factor matrix" is shown below, divided into s 
those variables whl^ch it was decided to combine into, a "payoff 
factor"* index, those which it was decided to combine into^ an "input 
~ difficulty" index, and those .which it was decided to omit because 

f 
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they do ;npt\ load apy more clearly on one factor than on \the other. 
The scores! 'are MLhe regression -weight* , of the common factors. (See 




Table - 5^". f<W^cod|%ete- list of the words used on the/'TTRH^iual 



semantic differential scales). 

v * . "V • . ^ , • 

Variable Factor 1 Factor2 

PAYOFF factor- • 

ao'od « ' ._' . .73 - -39 



Stimulating • .86 ,. .16 • j» 

Productive • .78 .33^" 

Fun .68 .19\. ~ -w- 

• ';>&r ^ ' *~ '~ * 

v Input Difficulty Pacfw- '( INPUT) . 
..Frustrating 30 -64 . ~ 

Easy \ .16 .72 ^ 

- ' . ' v ; v 

(Related to Neither or Both- not used) 

Time-saving ' -59 , .44 . • • « 

Friendly ». -46 ^.32 

"Demanding • * ..27 y .23 • 

Having Identified the INPUT and PAYOFF factors, an index .was 
- constructed by- "adding together the- scores for the component 
questions. * These . were then used In two stepwise. multiple . * , 
regressions, with several predictors entered In $rder to determine 
which ones ar| the most powerful determinants of these dimensions of > 
subjective satisfaction (See chapter two for an explanation of the 
nature ,and purpose of*stepwlse multiple degressions), The results of , 
the regression for* PAYOFF satisfaction are shown In Talfte 5-25- It is > 
-most unfortunate that only A™ .oases '•- had data for all* of these 
variables. . The small 'number "of cases makes It difficult »to obtain 
' statistically significant - results. 

The variables to' enter'into the multiple regression were selected by^ 
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first computing bivariate Pearson's correlations and significance 
levels ' for the relationship between the two indexes and sevetW. 
possible Jbredictors. Group, previous' experience with terminals, and 



possible predictors. Group, t previous' experience with terminals, ar 
satisfaction with the group leader were eliminated because tfhey did- 
not yield significant correlatiofts . The variables which were related 
\are shown below. * 
fOWN* "Number of group members known 'before system use 



ESTUSEf estimated' number of hours of use per week, before using EIES 

NUMBER 3 Number of persons with whom tpe user ^elt A'P^ active 
communication on EIES at follow-up < 

EIES MET= Number of these persons "met" on EIES 

HRSUSE= Number of hours spejjt on lijie at time of follow-iip, . 

Bivariate Pearson's Correlations 



:ases shown in parentheses) 

Variable PAYOFF INPUT 

KNOWN .15 .21 . ) 

* '■ * (47) {5ir\^/ t 

• p-.17 P 3 .07 

ESTUSE ' * . .30 .21 

(49) (54) v 

p*.02 p».07 



NUMBER .31 .17 

. (SI) (102)' 

«. at 



p 31 . a! p*.05 

EIESMET PS* ^.35* .15 

.(97) (106) 

P*.01 j)*.06 




HiSU"SE» • .33 .24. 

(98) (107) 
p-.Ol p».01 



The stepwise analysis shows that the most -important* determinant of 

"PAYOFF" satisfaction is .the attitude toward the system bfcore using 

♦ 

it / as indicated by estimated hours ^of use — once again, we 6ome uj^ 

with tjie finding that users somehow knew before communicating on EIES 
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how much they would, like the system and how much they were likely to 
\l< use it and'benefit from it; , t As shown in the -stepwise regression 
correlation matrix ' in chapt^tv two, the strongest observed correlate 
of p reuse estimates of EIES-use is the number of group members known.- 

* • . 

• The variable entered at Ihe second step, which significantly improves 
the prediction, is the number of persons f ^met" on EIES. ' A third f 
variable which improves the prediction somewhat (significant at the 
.yf lever but not the .05 level) is the number of persons with whom 
W Is communicating on EIES. In sum, our most important 
determinants of satisfaction with what' one is getting by using SlES 
are measures^ of /Social connectivity. Once these ^variables are taken 
into account, tj^me jin line has no independent effect. 

4 

None of the' variables were significantly related to INPUT 
satisfaction. The strongest predictor Is' the number of members known 
before using EIES. If one knows many other group' members, one Is not 
likely to feel that using the system Is frustrating or difficult. 
The second most powerful predictor Is the number of new persons met 
on EIES^ Even thoagh these findings are not based on enough cases to 

yield statistical slgnlf icanceA they are rather fascinating one's 

reaction to trying to use the-' system actually seems to be "determined 
by social factors — (number,* of old and new communication partners), 
and is not at all a product of non-social factors such as previous 
use *of computers or computer terminals, or number of hours of 
experience using the, system. 

Just for curiosity, the variable "EIES is Not demanding or intrusive- 
demanding or intrusive" was correlated with the same set of 
predictors, since it was furthest away from either of the other 




factors. *The most tmportant^4etrterminant of this subjective evaluation 

is the group to which the user belongs (significant /at the .05 

level). This finding fits in yrith our previous observation that 

things most 1 and least liked about EIES correlated with group 

membership, r ' i 
f 
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PAYOPP 

KNOWN 

ESTUSE 

NUMBER 

HRSUSE 



f table 5-25 
Stepwise Multiple Regresslqn 
Determinants of PAYOPP Sat Is'f action* Factors 

(Note: See text for definition of variables) 

/ 



CORRELATION MATRIX 

KNOWN ESTUSE NUMBER EIESMET HRSUSE 
' .14 .33 .22 < .26 .24 
.20 ' .47 ; .13 .^1 
.16 /El -''.46 

.44 .38 
,40 



D 



N of'cases» 44 



ESTUSE 
EIESMET 
NUMBER ' 



STEPWISE MULTIPLE REGRESSION 



FACTOR ,MULT R R 



• 33 
.38- 
.40 



SQUARE 
.11 

^ .16 



BETA 

-.28 

-.15 
-.10 



.01 
.05 
.10 
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SUMMARY, CONCLUSIONS, \HD SPECULATIONS - # 



1. Learning time Is a problem wi'th EIES. Although the reported 
median * for learning th£ .basic mechanics Is an acceptable 2*4 hours, 
mbre than a third of the "users have never mastered the advanced 
features. Perhaps, however, many of -triem* do not wish to,.* perhaps 
^they are quite content with a relatively simple set of capabilitil 
to accomplish their communication objectives on line. 



/ 

! 

2. Although the written documentation (manual) -is given generally 
good ratings, many users will not read through such lengthy printed 
material. Moreover, the -standard introductory manual does not cover 
advanced or new. features. An on-line explanation file, which is 
complete and up to date, is" hundreds of pages long. Although one can 
/search for and retrieve information on Just t those features or 
ca^jftilities of interest '(like turning to the appropriate page in a 
printed 'manual),- it seems to intimidate many users. Among the 
'variety of plternative source's of "on-line" help and documentation 
provided, the most popular is.Jdie human user consultants. 



3* EIES users 1 behavior ,and opinions' support the design choice to 
provide a variety of ^alternative interfaces, with menu4 presented 
first. 

4. U3ers are most likely to name "as "the most valuable feature" of 
EIES not . a software feature or capability but rather general 
characteristics or benefits of the medium, related to the people who 
use it, .such as "diversity of discissions" orjl'sharing of ideas." In 
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dis'cussing "useless, distracting or out of place" EIES features, 

1 

there are frequent complaints about „ slow system response time, the 
editor ,_y&nd the, difficulty of remembering the various commands and 
procedures f or^interacting with the system. However, the single mo3t 
frequent category of complaint relates not to the computer system 
features but to the behavior or quality of performance of those with 

whom one is communicating:, /"Junk mail," "cute remarks," "useless 

i 

material" entered, etc. 

*. 

/ 

5. One serious adaptation problem for users of this* medium is 
"information overload." About pne in five users "always" or "almost 
always" feels overloaded with material pouring oi^f of the system, and 



the majority f^el this way at least "sometimes. V^However, feelings 
of "information overload" peak at middle levels of experience, and 
then decrease markedly, even though the users with the'most time on 
line are objectively handling greater amounts of Information. The 
most experienced users have learned how to cope with the rich but 
potentially overwhelming plethora * of materials available to them on 

line. How they prevent "information overload" at high levels of 

\ 

activity on line J/s an important topic for further study. 

6. When the terminal is used at home^ other household members 
frequently' develop strong po'sitive or negative attitudes toward the 
system. ' 

\ 

s 

-7. Most users do not generally feel able or willing to pay the full 

/ 

cost of using EIES (more than $100 .per month) out of their own 
pockets. 



b. Multivariate * analysis indicate^ that the most important 
determinants of overall subjective satisfaction with communicating on 
EIES are aspects V social connnectivity : how many system members one 
knows before signing on line, how many people one actively 
communicates with though the • system, how many valued new 
relationships are begun with people "met" on EIES. 

Pernaps these systems are like parties. The software JLs like the 
refreshments, furnishings and decor. They , can help people to enjoy 
themselves and communicate easily, or they can detract from the 
occasion. 3ut the main determinant 'of whether it was a "good" party 
is the people there and the "quality of the social' interaction ^t the 
party.' The party may -be held in a mansion and catered by Julia 
Child, but if nobody talks to you? you don't *like it. On the other 
hand, the party may .be held sitting on the floor and feature beer and 
pretzels... but -if all your dearest friends and most valued 
colleagues are there, you will have a woijjperful tt«e. 



* • • ' Chapter 'Six 

IMPACTS ON THE SCIENTIFIC~RESEARCH COMMUNITIES 

How did the , usf of EIES Tor approximately lb months, affect the 
scientific research communities? In terms of the intellectual and 
social structure of the group and *its ties to, other research 
communities on and off EIES, what happened to communication patterns, 
cohesiveness, and -perceptions of competition irt the field? And, most 
importantly, did EIES in fact help to clarify or resolve theoretical 
and methodological controversies, in the various specialties, as was 
hypothesized? These questions will senw^ as the focus of this 
chapter. , Impacts^ or effects that might be generalizable to any kind 
"of user group, not Just scientific research communities, will be the 
subject of the next chapter.. Such more "general" impacts Are changes 
in amount and type of communication, effects on productivity, and 

ft 

general effects on- the way that users work and think.' 

METHODOLOGICAL PROBLEMS AND PROCEDURES 

At the end of 18-2*- months on EIES, there-had been considerable 
turnover in the composition of many of the scientific groups, with 
dropouts and inactives replaced by new members, a portion of whom in 
turn we«£-lnactive and replaced. Thus, even though the size and 
discipline of a group were the same at post use as «t pre use, it was 
a different group because the individuals belonging to it had 
changed. This is true for a longitudinal study of any., scientific 
research community, of, course, because healthy research communities 
have hew members Join and oldena^members retire or stretch tfeeir 

s > i 

Rir ' 241 ~ Z5Z 



energies into new specialties', however, the rate of replacement was 
exceptionally high for the on-line communities, 

f ' 

There are two approaches to the d^ta analysis • One gives us the 



largest number of cases 



to look at, permitting examination of changes 



within each group. This approach is to -compare the distribution of 

V 

all respons.es to the same^questlonnalre item at two points in time. 
' The problem is that we cannot know fio what extent differences were 
produced because the scientific community " changed, or because a 
3omewhat different set of individuals responded. 



The second approach is , to examine the responses for only those 

t * 

\ 4 

individuals who answered the full set of quest ionnaires . We then use 
methods suitable for panel data. • 9 

For example, we can take those 4 individuals who perceived a great deal 
of competition in their specialty *at time 1 and see whether they 
perceived competition at time 2 as the same or less. However, since 
we have pre-use or follow-up questionnaire data for many EIES members 
who did not complete C post-use questionnaire, or vice versa, this 
reduces .the number of cases to a small number for most analyses, 
unf ortynately , resulting in no statistical significance. 

.We shall generally rely on the cross-sectional data de-scribing the 

research communities at two points in time, since this does not 

require us to eliminate so much data that the cases remaining are 

insufficient in number to reach any conclusions. The panel data will 

be discussed, however, in order to supplement the cross-sectional 

data w|th the available information about how specific respondents 

q changed their 'perceptions of their research specialties \vev time. 
ERIC 242. 
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• CHANGES IN SCIENTIFIC COMMUNICATION 

Table 6-1 shows that the majority of users of EIES report that they 
spend more -time communicating with other members of their scientific 
community as a result of use of the system. As would be expected, 
this is strongly related to the amount of time they spend on line. 
At - ' u fte lowest end of the % system-use scale, half spend less time 
communicating with their group than they did before it *was available, 
whereas, among the heaviest ElsS^vsers , 90S -have* inyfes ted more time 
in communications. Perhaps the most surprising aspect of the 
reported changes, however,' is Jaow little time some scientists ^spend 
communicating within their specialty community. For more than a 
quarter of the scientists who spend le^s than 20 hours on line over 
an eighteen to % twenty-f our month p^rl^^t^ reported to be more 

time than they would otherwise have devoted to communication with 
^rtelr peers. , 
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Table 6-r 



C 



Relative 


Time I 


gvestraent in .Communication with 


Specialty Group, 






6y Hours on Line 






A 
0 


• 






Hours 


Less 


More 


Same 


M 
IN 


c 

1-19 r 


r* r\ 0 


275 


235 














20-49 . 


435 


. ' 435 


;135 * 




50-99 


335 


635^*^ 


" \ 45 


24 


100± 


105 


905 


0 


19 , 


All ' " ' 


365 


535 


115 


99 . 






Source: Post-Use 


Questionnaire 






Chi square* 


21, p».01 





0 • 

Contingency coefficient* .42 ' 

-Question; Compared to the conventional means of communicating with 
your group, has EIES : ^ 

Involved less of your time. 

Involved more of your time 

Invo^V^d the same amount of *time 



i 



2o u 
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Broadening of Con'tacts rather than Encapsulation 




One question, a^ked at the beginning of this research project was 
whether the use of E^ES might not "encapsulate" the communications of 
its members ;withln the relatively tiny on-line group of peers.' Such 
a process wbuld have the negative effect of gradually decreasing 
contacts with /researchers in other specialties and thus impede the. 

r , 

valuable and fortuitous prodess of , cross-f efrtilizatlon of ideas. 

On the contrary , ^EI5S is more likely to broaden contacts with loca!l 

• « 
coll&agues, as system members become indifect' links between the 

oh-line and off-line worlds. Table 6-2 shows that, for three 

quarters of the scientists, access to EI5S has no effect on the . 

amount of communication with other* scientists in the specialty who do : 
• 

'riot have system access. There are 1 practically no reported instances* 
of a perceived decrease in communications with fton-EIES colleagues as 
a result of system .use. However, a significant minority, 

surprisingly even among those w ^o do not spend much % time on line-, 
report that communications with these colleagues has actually 

increased. The explanation is probably that they are serving to 

* 

relay information about and from the. system to off-line tftolleagues. 

Table 6-3 indicates that scientists using EIES are much more likely 
to report an increase in "communications with researchers in other 
disciplines or specialty area*" (43%) "ra-t her b than a decrease (only 1 
person, or It). There are no statistically significant diff ereffces 
- among groups for this finding, 4 though * the percentage reporting an 
increase did vary from only 17* for Group 54 to 545 for Group 40. • / 
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vTlrere is a moderately strong relationship with t^rae on line, as would 

*• 

be expected. ^Oiven our data on the large amount of electronic 
migration among groups and conferences that took place, most of this 
perceived' increase in communication acrdss disciplines is probably on 
line rather than off. 
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Table 6-2 



Impact ort Conwjunifation with Colleagues in. the Specialty but 



f 




• 


Notion EIES 




• 






By 


Hours on Lirie 






Hqups 


4 

Increased 


De$ 


ys»ased & No -change . 


. N 

<* 


1-19 ' 


"X .36*, 




r;;. 


i 


28 


20-49 


• - J 

24 % 


/ 




73%* 




5q-?9 


- - s * : 




- 4* 




■25 


* 100+ 
All 


32% 

1 ?5f 








19 
105 




Source: Post-Us3 (SUesiiiionnaire 



Chi -square* 7.2, p=.^0 



Contingency Coefficient- -25 



1 



h l 



Question* Has. the- use of EIES affected your Communication with 
the following? 

* \ ■ - 

Colleagues , ^n your specialty but not \ on EIES 

s - 

(Checklist- Increased, Decreased, ' No 'Change) 



* 
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TafMe 6-3 

Increase t»-Xl^mmunlcatlona with Resear.cher.s in Other Disciplines op 

Specialty Areas 



c 

uinulative 
Hours 


Increased 
■ 


Decreased 

• 


No Change 


N 

Responding 




30* 


0 


70 


27 


20-49 


3«S 


I 3 


59 


p- 32 


50-99 


H&% 


0 


52 


25 


100+ 


- 68* 


1 -o 


' . 32 


19 


ah ; } 


44* 




55 


103 



-n r 



Source: Post-Use Questionnaire 

Chi*Square=9.7, p^.iu 
• Contingency Coefficient= .29 
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PERCEPTIONS OP SPECIALTY GRQUP COHESIVENESS 



Table 6-4 Shows the distribution of responses by specialty group^ 
the perceived sociometric or network structure of i£he specialties at 
the follow-up and post-use points. Some* groups show^ a slight-, f 
tendency, toward perception of a less drer^ely knit, less connected or 
•Integrated structure, moving* to perception of isolated individuals 
from former perception of cliques or an integrated community. Group 
30,%utvire$ Research, and Group 54,-N.Mental Workload/ show this 
pattern most strongly. Tfris is the opposite of the effect that . had 
been.hypothesi^d". However, examination of the panel data suggests 
that the apparent changes- are attributable to differences in the 
persons responding. \ r - 

Looking at the panel data, there were, overall, no- significant 
changes - between the f ollow-up # questionnaire and . the post-use 
questionnaire approximately 1 year later in the extent to which 
specialty groups were perceived by. their members to be composed of 
isolated individuals, of cliques, or of an integrated group of peers. 
Most were likely to report the perception — i.e., cliques at post 
use — if they* had perceived cliques at follow up. ,Overall, the 
correlation (gamma) was .49, significant fet the .003 ^evel. , Among 
! those who .did £ive different reports, there was almoS't as likely to 
be a reported decrease in cohesion as an* increase. * F<?r instance, 
among the 6^who reported their group as a Single integrated research , 
community at follow up, 4 saw it as dissolving into cliques' or 
individuals by PQSt use. 'I Meanwhile, the total number of persons 
reporting 'an integrated research community increased from' b to 9* 
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Thus, "there is a tendency, albeit very weak,, for more persons to See 
the specialty as Integrated at post use. \ - 

m. 

The "^USn>ecs of persons are too small to produce any significant 
differences. In addition, the .measure is too gross to be.valW or 
.accurate. -Detailed measures yof connectivity within specific 
'specialties, such as those collected .for Group S5 and reported by 
Freeman and Freeman (19*0), may show some significant differences. 
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Table 6-4 

Changes in Perc.eived Cohesiveriess of 'the Research Specialties: 
^ a* Post-Use 



Group Individuals . ,• Clique's Integrated Responding 



» . Number 



30 
35 




61* 


; 33 


6 


3l% : , 


^ -50 


• 19 


45 %*~^ 


48 


7 


53% 


1 29 


18 


57% 


2 9' . ' 


14 


square 


- 7.3, P - -51 





18 
26 



40 45 48 7 2'9 

45 5355 .. '29 18 . 17 

54 ' 57* 29' . ' 14 7 



/ Follow-Up Questionnaire (About 1 Year Earlier) 

Number 

3roup Individuals Cliques Integrated Responding 

30 3555 50 15 ' 20 

35 3355 67-. ' 0 24 

40 -52S - 45 0 29 

45 / 3055 ■ 45 20 20 

50 6055 . • 20 20 - .5 

r 54 . 2555 M ~ 13 8' 



( 



chi square = 17.6, p 3 .29 
+ Follow-Up Questionnaire 

Question: At the present timj^ whlctr of the following best describes 
your ElfcS group? ■ m 

More a collection of individuals than a research community * 

A set o£ cliques or subgroups with interests . and activities in 

commtfn, but not an integrated community 

A well-inte^rat^d research community "that shares many 

interests and activities 'in common „ - - 



Changes in Perception of an Intellectual Mainstream 



In all ' except Group 45,' a larger proportion of specialty group 
members felt that there was an^»*intellectual mainstream" in the 
specialty after eighteen months of^ EIES discussions than before EIES 
-use. Whatever role \El^S^SS^ £*SVe played i'n olarifying' the 
•theoretical and' . met hodo logical V^rftroversies in the fields; it 
apparently led to a -tendency for some of the group members to f'eel 

that they were a little closer to recognizing a dominant "paradigm". 

i f * 

that characterizes research in their fairly new and interdisciplinary 

areas. The changes were not very large, however, and the reversal. in 
3froup 45 shows that' thi3 is a^contingent sort of development. As 
will be seea later, Group 45 is the one in which there was also an 
V^increase in perceived competitiveness and in which there were very 
^/few -perceived advances in clarifying specific theoretical and 
methodological issues. 

s 

/ * * 

aroup 54 is omitted from tl>is table because the smal^ " number of 
responses would make the'prejvs. pos't comparison invalid. 



) 

iuesti< 



'< Turning to the panel data] a T test was used on the question of 
whether individual scientists felt mofe in the mainstream or more 
isolated as the EIES trials progressed. This was measured on an 
ordinal scale on the pre-use and follow up questionnaires. The scale 
was : , 

1* Completely ^in the mainstream 
2» Somewhat in the mainstream , 



I 

3? Neither in the mainstream nor isolated 



O 252 

ERIC . 266 



! 



v Somewhat Isolated ' f , 

5» Completely Isolated 

» * 

* * 

There' were only 53 cases with b.oth pieces of dajfca. Surprisingly, 
**** there Is a significant shift toward feeling LESS In ime "Intellectual 
mainstr^pun" of the specialty. -The mean at time one was 1.3, and at 
time two, 2.3 (T- 4.78, p- -01). • This Is a^^th^r^ surprising 
finding, and one can only speculate on the reasons. Perhaps the 
on-line subgrdbp recognized Its on line discussions and conclusions 
as separating them from accepted % or taken for granted theories or 
priorities shared by the rest of the "off line" world In their 
specialty. 



Table 6-5 

Changes In Perception of an Intellectual Mainstream, 

by Specialty Group 

Pre Use 
Group 
30 

35 
40 

45 

Question: Is there a commonly accepted "Intellectual mainstream" in 
the specialty? 







Post Use 




yes 


N 


* yes 


. N 


54J 


13 


63% 


19 


27* 


22 


31* 


26 


33* 


30 


41* • > 


29 


71% 


14 


56* 


18 



Changes in Perceptions of Competition for^ the Specialty Groups 

Looking at the cross-sectional data, there was an apparent increase 
in the perception of competitiveness within the scientific specialty 
groups over the 18 months that they used EIES (see Table 6-6). 
However, the panel data indicate that there is a selection process at 
work as --fell as a change in attitudes among indidual members. 

The cross-sectional data In table 6-6 show a smaller number of 
members of most groups reporting low or non-existent competition at ' 
post use than at pre-use. However, this trend is not constant across 
groups. There was practically no change at all for Group 40, General 
Systems Theory. The most dramatic change was for Group 45, which 
started out with only 11* perceiving very intense or intense 
competition knd ended up with 43*. , 

The second' table on perceptions of competition (see Table 6-7 T shows 
that the changes ' are concentrated within specific' kinds of 
competition. There is a dramatic increase in all groups in perceived 
scarcity' of or codfretition for funds. Those who perceived unethical 
behavior among their peers dropped out .of EIES and did not respond to 
the post-use questionnaire, so this reason practically disappears. 
There is also some, increase in perceptibns of ' strongly opposing 
views. * >• ■• 

. However, the. panel data on the 45 X to 53 individuals who answered both 
questions indicate that the apparent changes in perceived overall 
level of competition are due to turnover in membership, with those 
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perceiving little competition within the specialty more likely to 
drop out.. On the question on overall degree or intensity of 
competition, the mean was 3^on the one. to five scale used, at both 
points in time. (T- \17, P-^)' For the specific kinds of competition, 
there- was a significant increase for the same types that are>pp.arent 
in the cross sectional data: competition over funds, perception of 
rival groups, and strong .opposing views. (For example, with "yes" 
coded as "1" ^dno check of a reason coded V as 2, the mean for 
"opposing views" was 1.7 at' pre-use and 1.4 at post use; T- 3-5, 
p-<.05). Thus, the conclusion derived from the cross-sectional data 
about increases in perceptions of specific kinds of competition with 
the specialties as an apparent result of discussions and interactions 
on the system does hold up with the panel data. 
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Table 6-6 

Perceived Degree of Competition by Groups: 
Post-Use Questionnaire ' 



Group 


Very 


Intense 


Moderate 


Low or 


Number 




or 


Intense • 




Nonexistent 


Responding 




30 




11* 


39 


59 


100* 


lb 


35 






65 


19 


99% 


26 


40 




its 


43 


40 


100* 


30 


45 




n* 


67 


22 


100* 


18 


54 




25* 


50 


25 


100* 


8 


All 




• 151 


53 


32 


100* 


100 



Source: Post-Use Questionnaire 



Group 

30 
35 
40 

45 
54 

Total 



Very Intense 
or Intense 

9* 
24* 
165 
43* 

0* 
21* 



Pre-Use Questionnaire 
0 Moderate Low or 

Nonexistent 



55* 
57* 
39* 
50* 
100* 
51* 



36* 
* 19* 
45* 
7* 

0* 
28* 



Number 
Responding 
11 
21 



82 



Chi square * 16.6, p « .03 
Source: Pre-Use Questionnaire 



Question: "How would you rate the "degree or Intensity of competition 
within your specialty?" 

\ 
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Table 6-7 

Percentage Checking Specific Reasons for Competition,, by, Group : 

Post-Use 



Group 



Funds 



Rivals 



Drive Unethical 



Opposing 
Views 



30CN-15) 
35CN-23) 
40(N«26) 
45(M-17) 
54(N*7) 

chi square 
P 



47J5 
35* 

82* 
57* 



9.2 
.05 
Source: 



20* 
52* 
38* 
29% 
57% 



50% 
61* 
35* 
59* 
29* 



' 5.6 5.4 
.23 .25 
Post-Use Questionnaire, 

Pre-Use 



7* 
0 

4* 
0 
0 

2.7 
.17 

N = 88 



53* 
35* 
58* 
12* 
43* 

10.3 
.04 



Group 



Funds 



Rivals 



Drive Unethical 



Opposing 
Views 



"30CN-10) 
35CN-20) 

40(N*25) 
45(N-13) 
54 0J-5) 



■185 
21* 
32* 
21* 
9* 



0* 
55* 
20* 
31* 
20* 



50* 
65* 
44* 
61* 
•20* 



10* 
5* 
4* 

15* 
40* 



50* 
30* 
36-* 
8* 
0* 



chi square 
P 



2.5 
.64 



11.9 
.01 



4.5 
.33 



7.2 
.12 



7.7 
. 10 



Source: Pre-Use Questionnaires 
number responding * 73 
Question: "What are the reasons for this competition?" (Check all 
that apply. ) 

Scarcity or competition for funds 
Rival groups of collaborators 

High achievement or success drive of people In the field 

Some persons act unethically. 

Strong opposing views % 
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Better Understanding of Others* Work 



The majority of EIES users, agree that the increased communication' 

with their. peers facilitated by using the system has changed' x -*te£lr 

understanding of the interests and activities of other scientists in 

the specialty. The more time they spend on line, the more likely \^t 

N 

is that such, increased understanding vill occur (Table 6-8). There 

* • C 

are . significant differences among the speciality groups in the extent 



to which' th^ process %occurS (Table 6-9). Such impacts are not 
related to our other measures of group success: the two groups in 



which there is the most "increased understanding" reported include 
one of the most, successful (General Systems) and .one of the least 
successful (Mental Workload). 

About half of the scientists report the related perception that use 
of * the system has changed their'Views of how their own work ^relates 
to that of others in the specialty C^able 6-10). Those who spend at 
least fifty hours on line are* most likely to report this perception. 
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) Table 6-8 

Extent to Which EIES has Changed Understanding of 
Others in Specialty; by Hours on .Line 

Agree Neither Disagree 

* or 

Strongly 



Strongly 
Agree 



N 



1-19 A~ 


3* 


45* 


17* 


Disagree 
35* ' 


29 


20-49. 


* 155 


42* 


36* 


•6* 


33 


50-99 


16* 


"fib* 


' 22* 


7* 


25 


•100+ 


21* 


53* 


21* „ 


5* 


. W 


All , 


13*'' 


48* 


26* 


13* 


106 



Y 



Source Post-Use Questionnaire 

Chi/square» 20.1, p=.02 
/ * 

gamma = . 3 4 

Question; EIES has changed *my understanding of % the interests and/or 
activities of others in my specialty. 



">27 



Whether ElfiS Increase<Xunderstanding of 

\ -- -t ' ) 

Other's in Sj>*cialty 



Group 



Strongly 
Agree or 



by Sp^l4lty Group 

/ # 
Neither Disagree or 

Strongly 



3 



* 


Agree 


m < 


Disagree 




?5 


nm 

— 655 


15 


■ 22 . 
..19 * 


100J-18 

• 

10O*»2.6 


40 


.^\77* 


.,23 >' 


. 0 


100*»30 


45 * 


" 37* 


42 


21 


> r iQ<i*»19' 


54 


• 86* 


14 


, 0 


/ 100*»7 


All 


61* 


26 


13. " 


100*»100 



V 



Chi square (computed qb uncollapsed data)*22.3, P*-G3 1 
/Source: Post Use Questionnaire . 

Question: EIES has changed my understanding of the ^%te rest s and/or 
activities of others in my specialty (Strongly "Agree> Strongly 
Disagree). * 



sr 



^7 



2Y<1 
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Whether EIES has. Changed Views' of How Work Relates to That^ of Others' 



1 


Hours • 
* 


* 

Strong^ * 


' Agree 


r Neither Disagree 
' or 

Strongly , 
Disagree 


N 

' 4 


0 




' 10? ..- 


, 21?. ~ 

- » 


21? ' 


48? 


29 




••20-49 - 

T „ 




36? „ 


30.? 


27? 


33 * 




50t.99 


• ' \. 8% 






16? 


25 




108V '■ 


^ , . '21% 


' 26% - 


32? 


21? 




0" 


AU ' 

* 


. * 10t 

» « • • 

i 


• 36? f 
Chl'rsTjtTare 


25? 

- 16.5, p».'06 


29? 


* 106 

#. 



. • . , Souroe» Pogt Usa 1 Questionnaire 

.Question: EIES has changed my view of how* my own work relates to that 
*<Jf .others in mjr. speciality. ♦ 

(Strongly £gree, Jfcree, ; Neither , agree nor' dlaagre?, Disagree*." 
""Strongly disagree) » * 



CLARIFICAjTJQN OP THEORETICAL AND METHODOLOGICAL CONTROVERSIES 

i. 

One of the most ' fundamental of the" issues related to the use of a 

technology such as EIES Is whether It -can speed the development ^ a 

disciplinary paradigm Or the process dt paradigm change when a 

•previous theoretical and methodological framework that Jias been 

dominating 'the field ^does not see.m adequate for answering fundamental 

a- • . • 

questions ..or guiding fruitful research. There are at least three 

. \ 

parts of this "process that .can be identified: formation of new 

.approaches', clarification of the nature of the differences between 

the old.' and the new approaches* and resolution of the controversy by 

some sort of synthesis or replacement, or through the d,emise of the 

« 

orooosed new approach, , - . 

♦ 

*; • * * * 

Overall, about half tff EIES users' ftelt that the use of the system had 

* * } . « 3 

clarified * theoretical controversies - within* the field. 1 It was 
generally not felt * that there had been a "great deal" of 
clarification, but' on£y that there had been "some*" Many of the 
comments accompanying this section of the post-use questionnaire 
pointed ¥ out that the controversies among r competing theoretical 
posi^ions^ had been clarified, but not resolved. The amount of 
progress on theoretical conflicts varied' by specialty,' with General 



n / 

Systems Theory (Group 4Q) reporting the- -most progress, and Devices 

■ / ' *' + ^ ♦ 

for"' the Disabled, a % relatively/ applied and non-theoretical 

discipline/ the least (see % Table 6-11),' would be expected, 

^naep^ion tof, clarification oJF theoretical - controversies is very 
strongly related . tO ( amount of : time spent ""On tine (Table 6-12% 

ivy^users-of the system f$ltV;hat this -was one 



/dlmost. ill off the*- heav 

r • : f • ; - 



outcome of their use of EIES, whe'reas'those who had spent less than 
an hour a month on line on the average "obtained no such benefit. 

T^ble 6-13 lists the specific theoretical issues which were named by 
participants in the various specialty grlups as having been 
clarified. Group 30 ' could not come up with very much specific. 
Group 40, which generated the largest percentage overall of perceived 
progress on theoretical issues, focussed mainly on tfce 'open vs. 

* closed system paradigms. m ^ ' ^ * • 

* Generally, use of EIES'was seen as somewhat less likely to have 

helped clarify methodological controversies" in" a Tc±*ntif*« s-peei*lty 

than theoretical issues. However, this was not true in all groups. 
Social -network theory-, members were more likely to perceive 

• methodological progress than theoretical progress, and named several 
specific methodological, clarifications. (See bibles 6-14 a*d 6-15.) 
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' ' Table 6-11 

Clarification of 'Theoretical Controversies, by Specialty Group 

Croup Great Deal Somewhat No N 

» 

• 30 (Futures) * ' 6% - 50 .44 13 ■ 

$5 (Social Networks) . 41 ' : 44 .52. 25 

40 (GenePal Systems)' 145 52 34 ' 29 

45 (Devices) 5% 16 79 , • 19 v. 

54 (Mental Wortfload) 0 ' 57 43 . , x 7 

-Ail -•/'..' " 7% 43 50 '</ 98 



Source; ?ost-Us^~^u^t ionnaire 
• Chi square^ 11.9, P=*15< 



7 



Question: Has EIES -helped* to clarify any theoretical controversies 
the specialty -area? 



-yes, £ great deal 
-yes, somewHat 
-no, ■ , 



{ ■ 

—. v 
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^ble 6-12 / 

WHETHER USE OP E&ES HAS CLARIFIED THEORETICAL CONTROVERSIES 

3Y TIME .ON L'INE - 






Great Deal 


Some 


No 






N 


1-19 hours 


05 


185 


825 




1005 . 
*■ . 


•68 


20-49 hours 


65 


385 


. 565 




1005 


32 


50-99 t hou-r-s- 


—&5_- 


525 . 


405 

• 




1005 


25 


100+ hours' 


21% ^ 


685 

• 


115 


. / 


1005 ' 


19 




Chi square = 
• 


26.761 


p = :ooo2 


gamma 


= .644 





Source: Post-Use Questionnaire , , ^, 

Question: Has S-IES helped-to clarify any theoretical controversies in 
the specialty area? . * ■ . 

yes, sL great deal 

yes, somewhafc - J ... 

no I> yes please explain .briefly the theoretical issue which jf. 
think has been clarified through EI5S discussions, and the extent .0 
which.it has been resolved. 



/ 
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Table 6-13 

• Nature of Percei-ved Theoretical Clarifications, By Group 
' . Group 30 (Futures Research) 

1. Cross Impact (expect a paper to be written) 
Exploring concepts of decentralization 

3. Subjective probability , m 

GROUP 35 (Social Network Analysis) 
. Clarification of differences J*n approaches $o structure. 

2. On the issues of- cognitive salience of networks, the conference 

has helped tjy expanding the controversy (getting opposing views out 

in the open) . » 

♦ * » 

3. We have a clearer idea of the areas In. which there Is diversity of 
conceptualization, and interprltat ion and where mort^work needs to be 
done. _ 

4. 0363 was/very useful, though It has not RESOLVED the problem of 
centrality measures. ? 

5y Homologlcal algebra may be useful? 



6.. The rdle and meaning of Atlcin's q-analy^is is being clarified. 

li ' ^ 

7. Concept of. centrality has . been 'Clarified, but not resolved. 
N GRoW ¥0 (General Systems Theory). 

( i, The open system/closed system detfate helped to clarify - the 
difference between general systems ^theory and cybernetics.. 

The- issue ha^ been raised but is still not widely understood. 

2. Open vs t closed paradigm. 

3. Sdfce agreement on terminology.. 

*\ 

, 4. ^ The "open/closed" paradigm debate has sharpened ' the issues 
involved. 



/ 



Table 6-1 J, cont. • 

5. Th^open-closed systems controversy; work on the glossary. 

6. ' The problem of large scale systera-f orbing;. Interaction with 
"information overload" problems. # 

Open and closed system pro*babiy obsolete as fundamental terms. 

7. System forming and system evolution. 

9. It has tried to identify the ^reas (set a boundary). The open 

system/closed system debate. j . . ! 

i \ • 

9. Self -reference. Open vs. closed systems. 

10. Forester model. 1 >. % 
11* Open/closed system. 

12. Open-closed system: 'differences e^ist. 

-13.., -S^op-e -of -f le4d 

■ V - 

GROUP 45 (Devices for the Disabled) # 

*1.- ?*ob.lem4 of marketing and commercializing devices- not resolved at 
all. ' '( 

2. The kind of information that needs to be sent to government policy 
makers.' f 9t 

GROUP 5^ (Mental Workload) 

1.. Information theory measures. Man-machine design, (not resolved) 

2. Definitions/limitations* 
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• • TABU* 6 j- 14 • • 

Clarification of Methodological Controversies, by Specialty Group 

No N (100?) 

18- 
26 

29 

6 

9^ 



Gr9up 


Great Deal 
i • 


Somewhat 


No 




0 


50 


50 


35 


I* 


46 


46 


40 


0 


-.- 3.1 


69 


i 

45 ■ 


5'S 


21 


74 




0 


50 


50 


All 


- 3* 


38J 


593 



1 Source: Post-Use JJiiestionnaire 
~ " Chi --square ti—fc*. 32 - *r - — ~ 

Question: Has HIES helped to clarify any methodological controversi 
in the specialty area* 



* 



-yes, a great deal 
-yes, somewh&t 
-no 
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\ Table 6-15 

Named Methodological Controversies which EIES has Helped to Clarify 

Croup 39^ (Futures Research) 

1. structural models- class^f icat ory scheme developed 

% * 

2. -cross Impact methodology 

5. The role of modelling . * 

^ 4. structural analysis 

Group 35 .(Social Network Analysis) ^ 

\ 

■ \ 

1. e.g., clarification of the topological algebra approach to 
structure 

2. My conference began to work on methodological problems in field 
work. Unfortunately we ran short of Hime. • 



3. Blocks and cliques- delineation and measurement are more clearly 
specified.\ , 



7. I3sue flf informant accuracy has been clarified, though not 
resolved. 



4. The ^ti^cjission about my experiment on EIES was very useful in 
helping toy define the issues 

5. ReaTXty of networks k \ .... 

6. Different programs and their best uses. 

3roup v 4G (General Systems Theory) 

1. Tlje discussion about DYNAMO was useful * - 

2. Systems dynamics discussion ^^-^ 

Group 45 (Devices for the Disabled) 

1. The need for a data base-^" 

2. How manufacturers design, implement and evaluate. We are getting 
more clinical. input . 

3. In terms of evaluation procedures 



Mi s 
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SUMMARY AND CONCLUSIONS 

V - C 

As a result of- using EIES for a period of 18-24 months, 

A. Total communication within the scientific specialty Increases. 

1. The majority of EIES members spend more time communicating with 
their specialty group colleagues than they otherwise would. } 

2. Three quarters report no change in amount of communication with^ 
off-line colleagues in the specialty. One Quarter report an 
increase. Thus, there is an expansion of 'indirect communication ties, / 
rather than an "encapsulation" of the on-line group. This lack of 
negative impact on communication with off-line collegues l£ an 
important finding. * 

* 

3. '.Almost half report, an increase in communication with scientists in 

other specialities or disciplines, and practically none' report a 

\ 

decrease. 

t / 

B. As a result of this increase in communication:^ 

1. There Is not an Increase In the perceived degree of Integration ' 
within. the specialities. At the end of the observed period of EIES 
use, as at the beginning, the specialities are generally seen as 
"collections of Individuals" or "sets of cllqtfs*," rather than as 
well integrated research communities. - 

• > 

\ 
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2- There is- no .change in the extent to which the scientists believe 
that there is an "intellectual mainstream" or well developed paradigm 
in the specialty. 

V / 

# 

3. There is a significant 'change in the extent .to which the 
individual scientists perpelve themselves as "in" such a mainstream, 
to the extent that one exists. The change is toward perceiving 
themselves ' as farther "but" of the mainstream. This is an unexpected 
result* 

4. There if a tendency for perception of .competitiveness within the 
specialty, particularly competition related to competing ideas and 
competition over funds, tp grow. 

5. There is an increased understanding of the ^interests and 
activities of other scientists in the specialty, and of how one's 
work relates to theirs. 

✓ 

C. With regard to the hypothesized process of clarification of 
theoretical* and methodological controversies, about hay the 
scientists feel that use of the system has somewhat clarified the 
nature of theoretical controversies within their specialities. Such 
perceptions *va^ significantly among.", the specialty groups and 
increase with time on line.- .Clarification of methodological 
controversies is less frequently perceived. .'Resolution of ^ the 
controversies has nop occurred. 



Perhaps the increased communication that occurs on EIES has effects, 
like those of a political campaign. One becomes more aware of the - 



V ( 

issues on wtiich there 1^ disagreement, and of the divisions within 
the (scientific) society,, And perhaps one needs^a structured process 
like an "election "~to resolve .these disagreements. 



The findings reflect the participants' judgments, that -EIES is better 
for generating Ideas and ' exchanging ppihifths than for reviving 
$11 sag re£to|nts. Ho^etar,- controlled * experiments indicate that it is 
possible to create structured processes of communication within the 
medium that do make it " likely that ' a gfoup will resolve .its 
differences and reaoh consensus. Either formal human leadership 
processes, or a decision aid based on systematic computer feedback on 
the nature of differences of opinion as expressed through formal 
; T voting," hav'e enabled groups on EIES to reach total consensus (See 
Kiltz, Johnson, and Turoff, forthcoming).- It ^would be interesting to 
see if future groups gf scientists could resolve the controversies 
which surface as a result" of. their computerized 'coramuni c'ation with 
the assistance of such special structures for generating consensus. 
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•CHAPTER, SEXEN 

IMPACTS ON COMMUNICATION' PATTERNS, WORKING PATTERNS, AND PRODUCTIVITY 



In this chapter, we look at the perceived impacts of EIES which might 
be generalizable. to any type of professional or .managerial user* 
There are four sets questions that the data presented address: 



1. How 'does use of a/system such as EIES change the amount of use of 
other -communication media? For instance, does it substitute for 
travel, add to travel by stimulating contacts with colleagues in 
other locations,* or have no effect on travel? 

2. Can social relationships and working relationships be formed 
and/or ^sustained* on an electronic communications network? 

. >. ■ 

Are there any general impacts on the way ijp which Knowledge or 

information workers (of which scientists are our example for * this 
study) think and work? , 

4. What are the impacts on productivity? This includes the provision 
/ot the, means to v higher " productivity,^ sifch as better access m to 
information and iSeas;. and perceived impacts on* the tptal quantity 
and quality of work accomplished. 



Finally^ w-e will use multivariate analysis' and a PATH diagram to pull 
together the model which emerges o^^e causal chain- of EIES use... 
from determinant* 'of amount of use 'of the • system^ through the effects 

of use on eorrciiunication patterns and productivity-* 

/ ' - 
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MEDIA SUBSTITUTION? * 



One possible expectation is that a- computerized conferencing system - 

w 

can SUBSTITUTE for communication via . other .media, taking the place of 
mall, telephone, or face-to-face- meetings. In the case of scientific 
communities, information exchange* on line might conceivably evten 
substitute for book or Journal reaflng, in the sense., that the time 
invested in reading papers, and conferences - .on line might *be 
subtracted from some fixed y total amount^of time available f or~ 
Tr keeping up" *ith the professional literature in 'one's field. Under 
the substitution /model, one would' expect a decrease -in the use of 
othsr media. 



' Some of.tfte greatest hopes for economic viability of computer-based 
* communication systems stem from , the Idea that.lt may replace more 
expensive means of communication.. Nilles et "al- (-1976) 'focus on the 
ability to tel«commute to work' rather than waste" time and petroleum 
resources on daily * commutation to an office. Kollen's (1975)~ study 

.looks, at "travel/communication tradeoffs" mainly in t^rms* ^ of 
substitution for business trips' at -which' fa?e-tp-?ace meetings fcdke 
place, i ^ 

• * ■» 

One of the. stated ob-jectiyes for '€he use 'of message systems Is 
usually , to replace the,- letter or the' internal memo - or the telephone 
call. For instance, one- gQal/ Justification of the electronic, mail 
system- tried - dy Swens" Corning, as' reported *by McNurlin (19oO:2-3J ^as 
cost displacement through suJipRKutlon for rn^ii and telephone: 
" ' ... experienced, users typically replaced four to six } 



communications *a day, which, with, a future projected 
population in the. 'company of, 1$00 users, would show 
replacement savings of $600,000 a year/ / ' 



On the other hand, one •coulcf^Speculate 'that perhaps computer-based 
^communication may be ADDED ON to other communications, rather Vc^Ln 
substituting a new mode. JB)is may be particularly true with a/system 
that iqclixdes only a relatively small number of addresses or Members ^ 
with most of the people whom "one' communicates not available orv line. 
J§ne might un^r ' these cicumstances .maintain one ? s usual 
communications "channels but ad& on to them new 'Communication with 
people who 1 have not previous\ry-/t^een ^ easily .accessible. Under the 
add-on model .one would expect to see use of other communication modfes 
remain constant ("no effect"). 



A third' hypothesis might be^ termed communication EXPANSION. This 

model pictures the CC based coimi/nication being added on to existing' 

communications; and then stimulating more communication* via # other 

media. This might take the form of^elephone or travel or mails 'to 

supplement ^CC comunicatiorrwith people\iet on EIES, increased reading 

of boofcs or Journals due to discussions and refeVences^ encountered on 

line« of increased communication with -off-line » colleagues that is 

4 ' 
stimulated by ^system use. Under the expansion model, one would 

expect to see that*use of ottjer media' actually increases. 

. *. ♦ 

/ ' 

* 

Whether substitution, add-on, or expansion phenomena are observed 

V ' /' . 

will of 'course -be expected to vary according to the total amojunt of 
use made .of a system. At low levels 'of use, one^wbuld not^xpect it 
to affect other communications t very mych .one way or the otjier. It is 
probably the EIES u^ers vho spent a relatively high. amount of time on 



line (IOQ hours or more over eighteen to twenty-fo^? months) who are 
mo^t predictive of the potential media substitution effects, shoald 
such systems become widely used within an organization or 'intereat 
community. Thus, we will look at reported effects cross-tabulated 
by amount of time on line. To ? the extent' that significant 
* differences are observed among the user groups on EIES, it indicates* 
that mldia Substitution effects are also dependent on application 
(task, size and social cohesion of the group, etc, are all bound up 
indifferences among the grpups on the EIES system). 



\;here is generally 



In .TatfTes 7-1 and 7-2, we see that there is generally an ,f add on" 
effect in % relation to mail and telephone, but as system use 
increases, the "substitution" effect becomes more prominant. 
Overall, a quarter of all members and half of the heavy users report 
a decrease in the amount of use of the telephone, as a result of 
using EIES. However, a minority^demonstrate an "expansion" effect: 
14? overall report <an increase >n the. . use *of the telephone 
attributaole-to using EIES-, and 'this increase' is also directly 
related tV^rmount* of - use of the system. 



The pattern for mail is similar, only stronger., That is, at all' 
levels of system use, there most likely to be "no change" in the uss/ 
of mail as a result. But the likelihood of both reported decreases 
(substitution of CC fop mail) and of Reported increases (more mail as 
a result of system^ use)' varies directly as. a function of amount^ of 
ase of the system. Among medium to heavy users,, substituipn erf CC 
for mail ^ the modal pattern; but expansion also 'increases to 
'approximate equality in -frequency with the "add on" pattern. 
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A probable explanation. Is that on-line communication substitutes for 
some mail, or telephone but stimulates other contacts that might not 

i- 

take place otherwise. For instance, users- may apprise one^ another of 
available preprints or other documents, which are then sent by mail, 
, If a subject of mutual interest* is likely to take a, greaf deal of 
discussion, participants who find themselves on line at the same time 
frequently seem to decide *o talk it over on 'the telephone to resolve 
an issue ^or to get another set of c^res about one another's feelings. 
In other words, qualitative observations suggest that dyads resort to 
tHe telephone as a supplementary means of ^ommUnication' f or fairly 
long (ten minute or more) conversations, particularly if they ■ find 
one another on line at the.3ame time and are .thus obviously available 
to take a call. It is the heaviest users and those "who make tW% most 

3 

new contacts who are most .likely to exoand their use of mail and 
telephone as a result, of CC. 

.Tables 7-3 and 7-4 indicate 'that the prevalence* of substitution, 
expansion, or add-on effects related to mail or telephone is somewhat'* 
dependent on the group context. Among EIES users, nobody in Group 30 
reported- an.. iaaneas-e -in the use of mail- or telephone as a result of * 
using rftES, This futures research group had the largest, most active 
confer#bce, and thus a gfSwtt deal of 'group rather than dyadic* 
communcltlan, for whiife^Snail and telephone are most suitable. 
Paradoxically, Group 4,0 was comparatively^ likely to report both ^ 
increases and decreases Jn the use of mail and telephone. Group 45, 

Wh J»° h WaS an ,in ^ 0pmation ' exch ange about R&D and kind? of devices for 
**the handicapped, was , the most consistent in reporting decredj^Ts but 
not lndreases in the use of mail and telephone.' / 
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* * * 

Turning to" travel substitution, attendance at professional^ meetings 
.was separated from travel to -make a personal vi^sit with a colleague. 
Table 7-5 indicates that system use ,does not^ hav£ any significant 
impact on -attendance at prof essional society meetings* bOX report 

. "no effect", and those who do perceive an effect are almost as likely 

» » * * * 

to report an increase as a decrease, at all levels of system use.' In 
terms of travel for a personal visit, thererTs more likely , to be a 
perceived impact,' and -once again, ^such travel is about as likely -to 
•-Increase as to decrease. Among the heaviest users' of the system, 
almost a quarter report an increase in travel for this purpose. It 
' would seem,^ therefore, that as long as travel budgets are not cut, 

'contact Srith colleagues on line' is about as likely to stimulate 
"travel as to substitute for it. Anecdotal evidence suggests that 
among those who interact^ a great deal on line but have never met ^ in 

Jperson, there is tendency? for curiosity to prompt Extensions, to 
business or personal trips made for other purposes., in order to meet 
with one's on-line acquaintances. 

The' reading of professional b90ks and journals is much more likely to 
increase rathlr than decrease as. a result of using EIES (Table 7-7) • 
Apparently the .discussions with 6ne r s colleagues lead to more 
interest in reading journals, since the greater the- amount of time 
spent on line, the more likly it is, that such reading increases. In 
' Tabl-e 7-8, we ! see that , changes in ^reading patterns' %r*e group 
dependent as well as being .related , to time on line. -G'royrp- 54 is the 
only community in which a significant proportion report/a decrease in 
prof essionai^literature reading. -Group 40, which t^td a very lively 
.paradigm debate -on open, vs. closed"* system concepts, has the most 
members reporting an increase in reading. (The difference 'in overall 
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totals Is due to the omission of. Group 50 in these data; a 
disproportionate number of, these individuals report a decrease). 

In Table 7-9> we observe the perhaps surprising phenomenon 1 that use 
of EIES is more lilcly to increase than decrease communication with 
one's co-located (off-line) colleagues. Even more surprising is that 
the lowest level users are most likely to report an increase in 
communication with <r5lleagu^s within their own organization as a 
result of using EIES. Practically no one reports a decrease as a 
result of Using EIES in • communication with co-located colleagues. 
Perhaps the large proportion of low level use'rs. who report an 
increase iri local communication can be explained by their use of the 
system as a kind of toy which tifey occasionally demonstrated to 
collegueS as a curiosity or status symbol. Since we did not ask 
about the content of off-line communications that were increasing, for 

any of the modes, however, we can only speculate abou^the nature of 

* ■ 

.it. * 



In sum, for all modes of communication, low levels, of system use ar* 
most, likely - to have no effect on the use of other^ communication 



use is^\Jc 



mediae system use is^\J.mply added onto .existing ^communication. 
However, at high levels of system use, one is very likely zo also 
expand the use of other communications media as an adjunct to on-l^ne 
communications. This corresponds to reports presented in the 
previous chapter that, ^especially 'among those who spend^a lot of time 
on line, the total amount of time devoted to communications increases 

4 

significantly. * . * 
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, Table 7-1 

t Impact on Amount of .-Use of .Telephone, by Hours on Line 



U m mo 


Increased 


No" effect 


Decreased 


N 




IIS 






2b 


20-49 




«« 


,13? • 


32 


50-99 


24? 


52? 


24? 


'25 


100+ ' 


17? 


33? 


- 50? 


18 


All 


1415 


6355 


. 23? 

i 


. : 103 




Source : 


Post-Use 


Questionnaire 






...Chi 


square' 3 


16 i p- .01 






\ 


Gamma* 


•14 , 





^estidn": "Has the use of EIES changed the amount of your use of other 
media in the last year? (Media checklist with increased-* No effect- 4 
. Decreased, as choices) 



Table 7-2 

Impact on ^jujunt of Use of Mail, -by Hours on Line 



Hours j . 


Increased 


No effect. 


Decreased 


N 


1-19' 


11? 


68? 


21? 


28 


20-49 


■ 19? 


47-? , 


34? 


32 


50-99 


k 32?. 


• V 28? 


40? 


25 


100+ 


22? • 


\ ' 28? 


# '50? 


18 


All 


20? 


45? m 


. "35? 


■103 



Source: Post-Use Questionnaire ) 
Chi square* 11.9, P 3 .06 / 
Contingency Coefficient* .32 



Question: Has the use of EIES changed the 'amount of your use of other 
media in the jLast year? (Media checklist with increased- No effect- 
Decreased as choices) 
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Group 

Increased 

No Effect 
« 

Decreased 



Table 7-3 
Effect on Telephone Use, by Group 



30 



0% 



83* 



171 



35 



12% 



72% 



16% 



40 



24% 



4555. 



31? 



45 



11% 



56% 



33% 



54 
29% 
42% 
29% 



100% 



100% 



100% 



100% 100% 



18 



25 



29 



18 



Source: Postr-Use Questionnaire 
Chi square = 10.5 " P = .10 



Question: Has the use of EInS changed the amount of your use 
othgr media" in the -past year?- 
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Table 7-4 4 
Effects on the Use of Mail, by Group 



•Group 


Increased 


No 


Effect 


y Decreased . 


N 


. 30 * 


. * 'ox 




67 


33"' ' 


- 18 < 


35 


2«X 




. 40 


« 32 * 


.25 


40 


3756 


■ 7 


30 


' 33 " 


. 30 ■ 


45 


' 12% 


35 


! . 53 ' 


17 


54 ' 


14% 




43 


43' 


7 


All 


22% 




41 


- 37 


97-. 



Chi 'square=13.9, P a .0« 
Contingency coefficient^ .35 •' 

Question: Has the use of EIES changed the' amount of your use of othe 
media in the last year? • ■ . s 

(Checklist included -.Mails- Increased, No effect, Decreased) 



' ( 




\ 



2SC 



\ 
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Table 7-5 

Impact on Amount of Travel to Professional Meetings, by Hours on Linfe 



Hours 


Increased 


No effect 


Decreased" 




1-19 


. .7* 




10% . 


' 29* 


20-49 


7% 


8155 ' 


' -13% 


31 1 


50-99- 


12% 


88* 


"V 


25 


100+ 


17% . 


61% 


' . 22% 


18 


All 


}D% 


80% 


11% 


103 



f 



. ' Source: Post-Use Questionnaire * '* 

C£i square^ 7»7, p a# . - 26 " 
Contingency Coefficient* .26 

Question: Has the use of EIES changed the amount of your use of other 
media in the last year? "(Media checklist with increased- No effect- 
Decreased as choices) % . 

Table 7-6 * '" S ' 





Impact on Visits 


•with Researchers In Other 


Locations , 




> 

# 


By Hours on 


Line 




Hours 


• 

Increased 


No effect ' 


Decreased 


N 


1-19 


n% 


82% 




28 


20-49 


13% 


69% 


19% 


32 


50-^99 


.8% 


» 

88% 


4% 




100+ 


' 22% 


50% 


'28% 




All 


1-3% , 


74% 


.14% 


• -103 • 



Source: » Post-Use Questionnaire * 
* Chi square- 10.1, p 3 .12 
Contingency Coefficient- -30 

Question: Has the use of EIES changed the amount of your use of other 
media in the last year? (Media checklist with increased- No effect- 
Decreased as choices) 
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» « 

Impact on Reading Journals or Books, 



• 


• 


By 


Hours on 


Line , 


r Hours 


Increased. 


No 


effect 


Decreased 


Mf-19 . - 


. 17* . 


» 


75? 


I 


i I* 










••' 25? 




63? 


> 13? 


' 50,99 


. 32? 




64? 


4? 


100+ 


44? 




, 39? 


'17? 












All * 


"2a? • 




62? 


• ro? 



N 
29 
32 
25 




Source: Post-Use 'Questionnaire 
Chi square= 7.9, p=.24 » 
Contingency Coefficients .27 v 

\ t 

Question: Has the use of EIES changed the amount of your use of other 
medlteln the last year? (Media checklist' with Increased- Mo effect- 
Decr^ped as choices) , 







Table 

t 


7-8 


> 


Change in Amount 


of % Reading of 


Journals 


Group' 


Inc/eased- 


No effect 


Decreased 


30 


22? 


78 


0 


35-. 


36? 


60 


4 


40 


40? 


57 


• 3 


45 


6? 


• 94 


0 


54 


29? 


43 


29 


All 


• 29? 


67 


1 



Question : 



Chi square=20.«b, p=.01 
Contingency coefficient -.42 



N 
-18 
25- 
30 
18 
7 
98 



Has the use of EIES* changed the amount of your use of other' media in 
the last "year? ' . ' *" 

> i . 

(Reading Journals or books... Increased, No feffect, Decreased) 
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Table 7-9 ^ s, ; * w f / 
Impact on Communication with Colleagues at 0ne f s\0wn Organization 

By Hours on Line - . ' v * 



_ . , — 

/"Hours 


Increased 


Decreased 


No 


change 


; > N" ' - 


' 1-19 




0 




58* * 


v. 

^ -28, ; ; 


20-49 


15? 


61 




',79% - 




50-99 


207. 


a? 




. 72% - 


100'+ 


167. 


' .0 






1SL 


All 


243 . 






72? 


105 

\" 




i 

Source 


i 

: Post Use 


Questionnaire 





Chi squar.e=10.a, p=.09 

Contingency Coefficient = .30' 

Question; Has " the use of EIES effected Jrour communication with any 'of 
the following? „ • 



Colleagues at your rinstitut ion or opganizat£on. 
(Checklist- Increased, Decreased, No Change) 
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INCREASED CONNECTIVITY 



There are .many Indications that the -use of EJES expands the size and' 

— denetey— of ,s^^anl~jaettfX)rl£a . Jy _size A vre mean 1 ' the - total number i of « 

s * * — s v 

person? with- whom qn'e is directly or indirectly in contact, and with A 
^^ft^om onVs^n fairly' easi.ly exchange infornjatipn and Ideas or' more- 
personal ' communications. By density, we mean the number of-' 

connections - within the social network.- Density is .defined 
mathematically as thpS . actual' number of ties among pairs in^a network 
divided by the ''total possible number of tiesbetween pairs- So, for 
instance, a density of .50. would mean that half of the pairs in the 
, social network or group are connected. Another concept is intensity 
or* multistrandedness of relationships. There are many kinds of -ties, 
from "knowledge or awareness of one another to close* personal # 
friendship. It is hypothesized that systems such as EIES 'can 
increase the intensity or strength of tifes as well as the size* and 
density of networks. . Such Aarge, densely knit networks with marly 
rich (multi-Stranded) relationships are potentially a very fruitful 
social setting for scientific progress or othec kinds of "knowledge 
work." 




In Table 7-10, we see the only questionnaire data Available for all # 
groups on EIES measuring growth in social networks. ' We see that most 
EIES users report -that they have actually met ar^d gotten- to know 
other scientists over EIES. As would be expected, the number oJL-new 
social -ties established on BIES Is highly correlated with* the amount 
6f time spent on, line. .Among ""those whb had spent 100 hours or more 
on line', a third had expanded tttei.f social/scientific network by 
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eleven, or more new persons. * . * ^ -\ 

Ii(f 4 an analysis / bein/' carried . out by Ronald Rice, we will look at 

changes In size and 'density of social networks on EIES over time- by 

♦ « 

lng. the who-to-whom data for message Absent . Until that is 
orapleted, the ';Jfcniy oth$r direct . measures'. "We have (other than . 
qualitative reports 1 of greater connectivity described below) are 
from^ a detailed "study of one of the EIES groups (35, -social networks, 
not 'by chance); carried out by the principal lftvestlgator for that 
group afip his coauthor /wlf e. * 
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Table 7-J.O ° 

Number of Persons "Met on EJES,* by Hours on Line 

Hours • • • None 1-5 6-10 11 or more , N- 

1-19 52% 35* * 10% 3% ' 29 

20-49 27% , 3.7-56 lf% , ■ 20% 30 

50-99 20% 48% ■ 20*' • - 12% . 25 

100+ • 6% 50% .11% . • 33% 18 

All 28% 41% 15% * 16% / 102 



* 

Source: Post Use Questionnaire 



/ 



* Chi square* 23, p = .03 
gamma 3 .38 



-Question... Of these, how many have you "met" (gotten to know) over 
EIES? ' . * 



* 
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/ The Group 35 Social Networks Study 
» • 

Twenty-nine .members of this group completed \an on-line version of a 
-social network^ questionnaire at the start of the . experimental 
period, and a* mailed' version seven months ' later. An interesting 
aspect of this study was 'that 21 of the 29 had 1 attended a fcwoAiay 
face to face 'meeting just before they completed the * first 
questionnaire. TWus it is somewhat amazing that a little less than 
half reportld ever having ff met n one another. 




There Were four ty-pes or levels of intensity of relationship asked 

for at the two points in time. Each participant was asked to 

designate those "they had heard of or read publications by; those they 1 

had met^ or exchanged' letters or phone calls or computer' conferenced 

with,* those whom they considered "friends," and finally, thos.e whom 

they considered "close personal' friends." Table 7-11 shows the 

density for these four levels of relationships at th^two points in 

time. WAs the Fr.eemans summarize the results, t ^ 

I The , data for the^ second questionnaire show a 
* considerable amount of consistent change. -There werp 

noticeable* .increases in the proportion of -people 
, reporting relationships of all- four kinds. It woift'd r 
seem that thrf computer conference, ^or perharps some r 
other events that took place during that seven month * , 
period, brought these people closer together (Freeman 
and Freeman, 4980, p. BO) 

s 

r . — ^ 

The" analysis also goes on to measure distance or "reachability" among 
Group 35 ' members.- A person is reachable if one is linked directly 'or 
Indirectly through several ties t ( such- as friend's friend). Distance 
.is the number of links required to reach someone by the shortest 

... N v 
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route.- For Example, my friend is one Alnk away, my friend's friend 

is two links (or d = 2} away; They found that the number, of 

reachable pairs grew whenever possible (when it had not>' already 

reached \00% for the 812, possible pairs), and that the distances were 

shrinking' on 'all relationships except ' those of close personal 

friends. The' Preemans conclude' th^t the network waa-cijanging from a 

clique structure to a genuine community (ibid): 

Por close personal friends, data from wie .first 
questionnaire seemed -to "show- the presence of i^ight 
little cliques,; by the ti°tae of the administration of 
the second questionnaire --there were many more per^ona^ 
friends reported and trhey were beginning ' to Be loosely 
/ . linked together ^nto larger structures. This suggests 
that at the end' of the second- ^questionnaire there was 
much more of a '^community" among these social networks 
' people. „ 
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Table 7-U1 . 1 f , 

Density of Pour Types of Social Relations' Bef ori and A;£ter 
Seven Months of Using EIBS, 'Group 35 

- ^ TIME 

1 - 

Relation t First Second 

\, 

** 

Heard of .62 .77 

Met > .49 ' *6S 

Friends .14 " .22 

, Close' friends , .05 .06 

Sourpe: Freeman and Freeman, f980, p. 79 
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Table 7-12 

" Average distances between reachable .pairs and number of 
reachable pairs* for: four relations at two tjimes, Group 35 

TIME 



Relation 



FIRST ' C ' • 

Distance No. of Prs. Distance No. of -Prs. 



SECOND 
< < 



Heard of 1 



Met 



Friends- 



1.38 
1.52 
N 2.7b 



812* 



* ai2* 

0 



72b" 



1.17 
1.30 
2. lb 



812 
812 
812 



Close friends 



2.01 



96 



3.13 



!2 V 



Source: Freeman and freeman, 19S0, p. SI 



IMPACTS ON THE WAY IN WHICH MEMBERS THINK AND WORK 
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An open-ended question probed the extent to which the use of EIES had 
"any 'Impacts on the*way in which you think 'and f wqrk, in general.^ 
Resppndents were asked to check yes or .no, and then to "describe 
these impacts in as much derail as possible." ~ 

i 

Overall, 52S report general impacts on working patterns, with many 

/ - - 

V 

describing them (sa^ .Table 7-13*). 'There are no significant' 
difference^ among the specialty groups,, with the proportion reporting 
such general impacts ranging from 40% dn 4 Group 30 to 71* in Group 54. 



The^jlescriptions of general impacts on the way in which members think 
and _work fall into\four broad, somewhat overlapping categories. One 
ha^ to do with broadened professional p*ersoectives or activities. The 
second relates to increases in communication or 'connectivity . ■ The 
third refers to a kind of change in perspecive about ttie relation *of 
self and cosmos caused by* the communications medium: ".disappearance 
of space and time are frequently mentioned aspects of this; >And the * 
fourth relates to specific work habits', such as being more organized', 
working at home more, ands4ncreased* pace o^work. 

General ^^jq^&cts by Time on Line 
As would be expected, impacts on ,the way one works and thinks are' 

c 

more likely the more, one use^ a system such as EIES. Reported 
impacts increase steadily from 39% of those with less than twenty 
hours total experience on line to. 78% of those with *K|0^ hours or more 

of >on-line time. 
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•\\ Tabid 7-13 u 

"V ' Impacts on the Way Users Work and Think 

\ \ (Quotes from an Open-Ended Post Use Question) , 

A» Broadened Perspectives • 

It has broadened my perspectives on my own work* and on the 

i environment In which I am working, I have been exposed to ideas 
' which JL would not otherwise have "encountered and \have been able 
to participate in more wide-ranging discussions .than* ever 
before. • I will -miss ^the lntelletftfual stimulation; the diversity 
of ideas, and the immediacy *qf communication. 

Much more opportunity toyffiseuss basic intuitions, perspectives and 
'opinions on what is /Valuable in this field of research. - My pwn 
work has broadened a great deal as a result, j 

I have been exposed to (1) a variety'oP people .in mjr research area 
previously unknown to nfe* (2) people in other research areas and 
their ideas, about the world (3) I have been able to ask for help 
from leading members -of my research community about current 
research problems. 

It has made me more. aware of\the issues which* some people in the 
field consider important; this has included some ' surprises , 

3roader exposure. to ideas. More aware of controversy within 
disciplines Familiarity with people in field. 

The ,worl& is larger than I thought — positively in that there are^ 
actually knpwledgeable ^people tfut there and (temporarily)* 
4 negatively in that there are so many with so many ideas**— that 
^temporarily) coherence and holding onto who I am suffer a bit, 

EIES has provided a richer VARIETY of information fo£ greater 
awareness of ■ u'rUversal/commort experiences of work done here, 
New'directions for future programming 

'I have become better informed about the details with which • 
\ individuals in an ancillary discipline are concerned, 

) A 
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B. Increased Communication and Connectivity 

# 

My first reaction now is to get on the system and get in touch with 
the appropriate person* I have been' doing more communicating. 

More informal contacts * " 

The instantaneous- feedback capability r of EIES in producing written 
material has had an outstanding effect on my work — being ori' # 
EIES and chairing a conference were very exciting to me.' 
Because of my age and my pre-PH.D. status, I'm sure' I would not - 
have had similar opportunities for several ^years . The non 
face-to-face aspect of. EIES enabled me tci present myself and, my 
ideas alongside -those of expert? in the field.- I am very 
grateful to NSP for this experience, cent Table 7-13, cont. 

Pocced me to be more aware c&L^nd take into account the work of a 
handful of social networKprts whose work is related to mine. 

I am more aware of many facets of scientific communication and have 
thought much more about information exchange. 

I am generally more* aware of people I out there who are at least J.n 
* sympathy with my broader research area. 

* « > ■* 

I can kick ideas around' among a larger circle of researchers .\ 

I have become addicted to instant gratification of- need to 

communicate. * I cpmmunicate more often on both important and 
trivial matters. 

C. Less Space and Time Bound 

Peel less time-bound 

Being on *-EIES is like being in another space-time; . I feel like I„ 
am simultaneously in Prance and in. the States, which has "befen a 
longstanding dream'of mine. ^ ** 

Sense of communication potential without time and space bounds. 

Expectation of the unexpected*4*ic£ease. , EIES enhances 'sense of 
-j value to be gained from spontaneous orders (see "Cosmos and 

TaxM f % F.A. Hayek). 

If iH&ve a tough question and little' time to answer lt K I'll ask 
people on EIES for opinions-- usually get one or -two-good - - 
responses. 
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Table 7-13, cont. 
D. -Work Habits ^ 

I spend 1-3 hours per day on EIES, usually in the morning, often on 
weekends and att night.* It hfes become "essential" to me. 

Greater appreciation ' of ASYNfcHROhlACY \n interpersonal ' 

communication... better time management in other non^EIES ' ' 
activities. ' ' 

I've noticed that trying t\keep track of half a down conferences 
has forced me to become nrore* organised in handling messages and 
general correspondence on 5IES< 

More time given to networks..AMore time given to 
reacting/interacting with^others, ovei? EIES. 

v 

I*hatfe subdivided my work more. * EIES is 1 , one element. 

It is a little easier to justify working at home to myself, sincS 
•the terminal is there. I can be at home and*"ab the office' 1 at 
the same time. 

I have had to learn to accept and live with what I previously would 
have felt was massive information Overloading. I have become 
.better at scanning and mucfi better at being precise myself, out 
*q? my recognition of the general feeling of information 
overloading that is perhaps the strongest impact of m entry into 
"The^Network Society". 

Need to organise information ftore efficiently. I ^structure my work 
using files\on local (UNIX) operating system. 

EIES is (slowly) forcing me to be "more aggressive- i.e. more 

on-line real time BEHAVIOR! (Less day-dreaming- more action!) 

The speed and pace of my work has increased due to quick feedback 
and ideas ffbm EIES. 

Using computer-mediated text editing, message services* and 

teleconferencing daily in my work. Local systems, national and 
international 1 systems. 

More aware of Importance of good. communications . Also more aware of 
need to screen out 'unwanted communication's. As an EIES PI, I do 
more and more of my work and professional .communications via 
EIES. , • f . 

Because of access to others, I can preview ideas more quickly than 
before. 
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Table 7-13, cdnt. 



• / 



Other 



I feel, and therefore act, more professional (a Ph.D' candidate) 

I have used the computer conferencing idea as one alternative for 
. proposing structured group, interaction on complex policy issues, 
The EIES -experience has made ;ne more aware of the pragmatic 
difficulties of implementing such a scheme. \ 

It has reinforced some of my own "germinating" ideas. 

Very much in favor of further* development of teleconferencing: 



SIES has- allowed me # to refine my thinking in previouly unclear 
areas. 7 



The potential is there, but a harcJ*copy terminal is very frustrating 

. to work with. A CRT is most 'important for scanning purposes. 
. * If I had a CRT', I would, be using EIES a lot as £ word processor^ 
<and it would increase my output by a lot?. 

Pr^vldeja a means of "assured" message delivery not always provided 
iby letter or phone. 

More\scientif ic models in therapy 

Makes you realize; how comparatively outdated conventional 
communication methods are. 
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IMPACTS ON PRODUCTIVITY . . 

AND" EFFECTIVENESS / 

Tables 7-14 to 7-19 show the distribution of total . responses t|o 
p'ost-use questions which probed I subjective perceptions of various 
aspects of the way in which EIES might have contributed to short or 
long-term scientific productivity or effectiveness. One "cannot take 
such reports ^at f^ce value; the respondents may have been overly 
generous towards' EIES* and inclined to see^.mprovements in their own 
work where more objective third ' parties would not. On the other 
hand, -the nature of intellectual work is such that only the person 
doing it is in a position to say whether something has helped or not. 

First of all/ we note that the system is somewhat more likely* to be 
associated with increases in perceived quality .of ' work thfen with 
quantity of work. The ways in which quality improved can be 

implied by reports <?f specific effects such as ltacre*asing the "stock 
of ideas," providing leads and references, and^ improving" conceptual' 
understanding. The latter ref ers • to shared conceptual space: 
improved understanding of the nature of -work beings done' by one's, 
<*p$ers and increases in their familiarity -with on'e's own. work. These 
effects ar* reported by about half . of all bsers* The largest 
percentages of reported productivity-related gains occur for 
increasing the stock of ideas and for providing leads 1 , references, or 
other information. i . 

j < 

Professional Advancement 



i 
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A separate class if items aaks\about professional advancement (Table 

T-19). There is little difference in' perceived impacts in the long 

term vs. the short term. But note an implicit tension between 

gen£ra3i scientific" status and" advancement within .the specific 

organization <by which- "one is employed. On the local, organizational 

\, / 
scene, one's.' connectivity to a national /scientific^ network is 

A I f r 

apparently frequently percei^d as imaging* immediate advancement, 

* \ ' 

Anecdotal evidence from users indicates that, some employers deeply 

s f 
resent these organizationally external contacts and efforts, and 

occasionally* even try to deprive-the employee of access,- 



V 



In terms of perceived "payoff" from EIES, another question shows that 
about; W% feel that they receive more than they put in and another 
-W feel that their, "payoff" is. . about equal to their contributed 
effort. \ 
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fable 7-14. 

Whether EIES Has Improved Quality of "Work, by Time on Line 



Hours . 


Strongly 


Neither 

llwl unci 


Disagree or 


M 




Agree 


or 




Strongly 








Aflcree 






Disagree 




W9 






• f' ' 










2w£ 


28% 

fa* w /w 




45% 


- 


25-49 ' 






32% 




44% 




cn on 

50-99 










28% 




100 + 

X WW » 




68*< . 


21£ 




11% * 


19 


All 




37% 


28* 




35% 


107 




< 


Chi 


square* 


13-5, 


p».04 


» 








Gamma = 


• 37 










Source* 


Post Use 


Questionnaire . 





Question: Use of EIES has increased my productivity in terms of the 
QUALITY of work recently completed or 'underway 

f Strongly agree, Agree, .Neither agree nor disagree, Disagree, 
Strongly disagree) ' 

Table 7-15 

Whether EIES has Improved Quantity of Work, by Time on Line 



Hours * 
* 


Strongly 1 
Agree or 

Agree 


-Neither 


Disagree or 
Strongly f 
Disagree ( 


N 


1-19 ) 


• 24%* 


17? 


59% 


29 


20-49 


15% 


• 38% 


47% 


34 


50-99" 


28% 


40% 


32-% 


25 


100+ 


53%_ 


• 26% 


• 21% 


19 


All 


27%' 


31% 


'42% • 


107 



' * - Chi square»14.4, p=.03 

Gamma* .35 ' 
Source* Post Use Questionnaire 

Question:) Use of EIES has increased my productivity in terms of the 
QUANTITY of work recently completed or underway 

(Strongly agree, Agree, Neither agree nor disagree, Disagree, 
Strongly disagree) 
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f^l ' Table 7-lb* 

Whether EIES has Increased "Stock of Ideas 11 fcrwFuture- Work 



Hours 


Strongly 
Agree 

• 


.Agree * 


Neither 


Disagree 
or 

Strongly 
Disagree 


N 


1-19 


10? 


55? 


7? 

* 


28J5 


29 


20-49 


151 


41? 


15? 


29? 


34 


59-99 


20? 


64? ' 


0 


16? 


25 


100+ 


32% 


58? . 


5? 

* 


5? 


19 


All 


1a? 


• 53? 


8% 


' 21? 


107 



Chi squSLre»13« 3, p««15 
Gamma = . 32 
.Source* Post Use Questionnaire 

Question: Use of ETES has increased my "stock, of ideas" that might be 

uasd in future work. ' • * 

(Strongly agree, f Agree, Neither kgree nor -disagpee, Disagree, 

Strongly disagre^r * " 

*■ * * t * 

* Table 7-17 

Whether EIES Has Increased Familiarity with One 1 s^ Work 

Hours 9 Strongly Neither • Disagree or N 

Agree, or * Strongly 
* * Agree .Di sagree 

1-19 4855 x 3455 ^ 17% >29 ' 

20-49 48 JC /3055 ' 21% 33 . 

'50-99' . 44%. 4455 * 12?- 25 ft 

100+ * 84JS ' 16* 0 19 . v 

All ,54% . 3255 1455 106 _ • 



Chi square- 10.9, p*.'Q9 
Gamma* .26 
Source? Post Use Questionnaire 



0 



WuesLionnaire / 

'Question: -EIES has increased the familiarity of others with my work. 

(Strongly agree, - Agree, v Neither agree nor disagree, Disagree, 
Strongly disagree) - • 
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Table 7-lb . *. * * 

Whether EIES has provided Leads,. References , or Other Information 

Hours Strongly * Agree Neither Disagree N 

Agree . # or 

Strongly 











Disagree 




1-19 


2b% 


/ 

41* 


14% 


17% 


29 


20-49 


24% 


49? 


12% 

» 


15% 


33 


50-99 


16% 


• 72? 


0 


,12% 


.25 


100+ 


58% 


37% • 


5% 


0 


19 


All • '* 


2955 


.50% 


9% 


12% 


106 



Source: Post Use Questionnaire 
■ Chi square*17.6, p.= .04 
gamma = -.28 

Question: " EIES , .has * provided me leads, references, or other 
information useful in my work. 
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^ . 'table 7-19 
Impacts' on Professional Advancement 



v 



Participation in EIES contributes to: 



Short term professional advancement in terms^ of my current employment 

Strongly* Agree 'Neither Disagree Strongly 

Agree Agree nor Disagree 

i Disagree 

6* 25% * 30% ,261 , 15* 



Short term professional advancement in 
pe6rs in my specialty 



m 



of my status among my 



Strongly 

Agree 

1% 



Agrefc Neither Disagree Strongly 

Agree nor Disagree- 
^ Disagree 
35? 37% 13% 9% 



Long term professional advancement with respect to employment 

Strongly Agree Neither e > Disagree" Strongly 

4 Agree — Agree nor Disagree 

/ Disagree y 

3% * 29% . 3755 19% UV 



Long /'term professional advancement with respect tojjy status among my 
peer^ln my specialty r 

Strongly Agree Neither * • Disagree^t'rongly 

Agree Agree nor • _ Disagree 

Disagree * 

7% 35% 19% 12% * 8? 
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Productivity Impacts'by Specialty Group 



The specialty groups dp vary in their perception of the extent to 

which EIES increases the overall quality of their work. * As woulcT be 

expected from other measures of Satisfaction with EIES,^Group 40 is 

.the most, positive. (See table "HfcO.) 



Significant differences do not occtir among the groups. % for increases 

-a 

in quantity of work. However, Comparable *dat;a collected for Group 
80, the hepatitis knowledge base project , shqw a reversal of the 
pattern of < answers for the .other groups. T}roup, bO, the only task 
related group. 1 has more agreeing that EIES increases tfce quantity * of 
work that they] are able to accomplish (nine out 6f twelve, or 75?) 
than agreeing that it has improve^ the quality of their work- ^five 
out of twelve, orA2X) f _ 



One might hypothesize^ tentatively that task relate'd groups are more 
likely to report overall gains - in "productivity (quantity or "quality 
of work) J than non-task groups. However, it does not follow that they 
do not experience as much gain in the "indirect" contributions to 
future productivity such as general increases in knowledge or id'eas 
or contacts. The third of the comparable items included for Group 80 
is whether EIES has Increased their "stock of ideas." Eight out of 
. twelve (67-JS) agree that it has, which is about the same .proportion 
that occurs for all the other"" groups, with the exception of Group 45, 
where only 37% feel that their "stock of ideas" increases. 
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Table 7-20 

Increase in Quality of Work, by Specialty Group 

■ N 

„ 100X-1B 
10055*26 * 1 
100**30 
100^-20 ^ 
100W 
10056 = 101 

Chi square=14.7, p=.06 
Source: Post Use Questionnaire 

Question: Use of EIES has increased my productivity in terms of th 
QUALITY of work recently completed or underway, , 

\ 

(Strongly Agree, agree,- neither agree nor disagree, disagree 
strongly disagree) ; ^ — ^ 



Oroup 

* * 


Strongly 
Agree or 

Agree 


Neither 


Disagree 

or ,, 
Strongly , 
Disagree 


30 


28* 


33 


39 


35 " 


•38Z 


a 


54 


40 


4755 


33 


20 


45 


2555 


50 


25 


54 


29X 


29% 




f All 


3655 


30 


1 35 



/ 
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Lack of Increased 'Publications 

We do not -have any 'reliable measures of the quantity and quality of 
material published before an<£ after EIES use. One m^Lght gather such 
data in the future by examining citations to their published one year 
before they began using the system and several years after their use 
of the syatete, since a duration of several years' lag time for t 
citations to that work will be will be necessary. 

We do have subjective reports by €he scientists for the number* of 
works of various types (articles, papers, text books, other books) 
"currently under way" and "published during the last year," at pre 
use and at post use. Such data are undoubtedly rather unreliable, 
but bne would ' expect , given the subjective reports of increases in 
factors related to 'productivity, -to also see a general increase in 
the numbers of publications reported. - This .does not occur; some 
counts go up,.somp go down, and mos£ show no significant differences. 
The one i^em for which there is a significant change actually shows a 
decrease, from a mean of 2\b tp a mean of 1.6 papers "currently 
underway."" <T» -2.08, p- <.0S,'!*» 80). There is not a sufficient 
number of cases to break these reports down by time on line or group. 
However, this negative finding does point up the problem of accepting 
at face value reports of productivity-related benefits. It is not 
possible with any of the ^available data to determine if in fa.ct there 
is any objective increase or* decrease in the quantity or quality of 
work accomplished as a .result of system use. It is certainly^ 
possible that productivity could actually t}e decreasing, if we had an 
objective measure. The active users spend a lot of time on line, and 
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perhaps -they would accomplish more if they spent more time directly 
producing and less time communicating. Certainly, a high priority 
objective £or future case studies is to develop more objective 
J productivity measures, to see if. the subjective impressions of 
increased productlrvt^y reported by heavy users pT such systems can b% 
Substantiated. 

- . ' • •• 
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MULTIVARIATE ANALYSIS 

9 * 

♦ 

How do the various relationships that -we have fit together .into a . 

« 

causal process? . * 

* 

* * 

Our first step is to construct an index that combines several of the 



dimensions of sub jectively r perceived increases in productivity. We 
will then build a model of the determinants of such productivity 
increases as a result of^aysteifi use; a model which is limited by* the 
variables on which we have data, but which is a useful first approach 
,to understanding the processes that occur. 

0 

Seven separate questions oft productivity-related factors are combined^ 
to form our index of the amount of perceived productivity increases, 
These seven questions are all highly intercom related, and a factor 
analysis shows ' thit there is "onlV ^one underlying . factor or 

dimension — lr\ oth*r words, they are all measuring different aspects , 

IP" 

of the , same thing. 'Our variable is called PRODUCTIVITY for short, 

and is formed by adding together the response scores (from one to 

five, 'strongly agree. to strongly disagree) on the fallowing items: 

QUALITY* "Use- of EIES has Increased my productivity In terrrfs of the 
, quality of wqfk recently completedStor underway." j 

QUANTITY* "Use of EIES has increased my productivity in terms of the 
quantity, of work recently completed or underway." 

IDEAS*' "Use of EIES has increased my 'st^ock (\t ideas' that^ might be 
used in future work." 

RELATE* EIES has changed my view of how my own work relates t\ that 
of others in my specialty." 

INFO* "EIES has provided me leads, references, or • other information 
useful in my work." 
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v. ' . » 

FAMILIAR* "EIES has increased the familiarity ' of otherS • with my 
work." 

UNDERSTAND* 1 "EIES has changed my understanding of the Interests 
and/ or activities of others in my ^specialty . " 

• * - \ 

A stepwise multiple regression was done using the three variables 

identified which ^adfvldually correlate most highly with both ih^ 

items in the index and the productivity index as a whole. » These are: 



TIME ON*^umulative number of hours oh line at post-use^ 
* * 

# C0MM« "How many different people do you feel that' you are actually 
exchanging information or communicating with on this system, 



currently 



9 « 



&IES MET a "Of these)^ how many have you 'met' (gotten to -know) over 



EIES?" 



*Jl1 



The resukts are .shown in Table 7-21. We see 'that the most important 
determinant of subjective Judgments * of a productivity increase ad a 
result of EIES use is how many new people one is communicating with 
on line whom one actually met through the system. Hours on line and 
the . total number -of persons whom one is communicating with also make 
significant contributions. Together, these three variables have a 
multiple correlation with perceived productivity increase factors of 
.54, meaning tha^ they explain 29% of the varian6e. 



Our ne^fr-^*t£p is to try to extend this analysis backwards to J*oin 
with earlier analyses of determinants of amount, of use of the # system 
to form a moder of th^ entire process which occurs <qn EIES. A PATH 
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analysis' was used for this purpose^ (See Duncan, 1966 and Ki*n and 
Kohout, 1975 J • A series of univariate and multi-varlate regression 
analyses are done 'to determine the strength of the ■ relationships^ 
among the factors, v/hlch are shown in tfcfe diagram in th£* f orm of the 
standardized regression coefficients (Beta), 

JJ^e model starts with our best predictors of amount of use of , EIES 

(see chapter two these are the number^of EIES members -known before 

using the system, and our conglomerate measure of pre T use motivation 
and attitudes' and personality factors— 4hd estimated number of hours 

•that will be spent on llne^each week) w . / 

The variables In. the middle of- the model are hypothesized as 
Intervening factors with both direct tnd' Indirect causal links to the 
Time 1 (pre us.e) variables and the Time 3' (post use) outcomeSv For 
Instance, estimated ' hours on line . has a weak but significant 
relationship with the number of people met on EIES. Time actually 
spent on line has both a direct effect on increased productivity, and 
an indirect* effect. Time on increases t*he number of persons met on 
line, which in turn Is our strongest* predictor of productivity 
Increases. The number of persons mat on EIES .als-o Increases the 
total number 1 of persons being c6mmunicated with on line, which is 
another direct determinant of productivity . ' " t/t 

.> * 

Not all of the possible Indirect links are shown, either becauae they 

* * } , 

are considered theoretically unimportant or unlikely and/or because 

emp'Jrically the causal link has no evidence. • For Instance, one might 

think that .perhaps the number of, persons met "on line Is affepted by 

the number knownSefore use, since one could be Introduced to new 

. ' 310 n 0 \ ? 
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acquaintances by old . ones.. However, there is no significant 
re'lationship. Likewise, one might posit that time on line alone 
increases* the number of persons communicated with, directly. In 
fact, there is no significant relationship (Beta- .03)\ • 



One * of the weakest links in the model is the determinants of the key' 
variable "EIES MET. ' Like - the initial level of estimated hours on 
line,,, this is probably, de^e/mined by a number of * unmeasured 
motivations and personality factors. .There are hundreds of potential 
new colleagues <on line on -a system like EIES; all -membens have an 
-equal opportunity to commtmica1^|with each other; yet some take 
advantage of this opportunity and some do not. Those who do meet 
many new people* on line are likely to be happier with the system (see 
Chapter f-lye) -.and__ to - perceive significant productivity Increases in 
their work. what determines the number of people whom one will reach 
out to meet and communicate with- dn* a system like this is a process 
worthy of detailed study. 
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Table 7-21 
Determinants of Increased , Productivity : 
A Stepwise Multiple Regression 
(Note: see text for definition of variables) 





FACTOR 


MULT R R 


SQUARE 


BETA 




eIes wet 


.47 


.22 


.39 




J 

TIME ON 


.54 


.29 


\ .25 




# COMM 


' .54 


.29 


.05 



P=13.2, p< .01 
N of cases 3 101 



s 



tec 



(J 
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A PATH DIAGRAM OF .EIES USE ' 
(Beta Coefficients shown in parentheses) 



3 



/ 



SUMMARY AND CONCLUSIONS • 



In terms of effects <?n media- use) EIES communication is -most likely 

..}-"*< 
to be added onto 'other communications; but thpse who use the system 

the most - are likely to ^also e*pand their use of o,ther communications 

modes. There- is some replacement «#f telephone and mails" by 

computer-mediated [communication. Travel to professional .meetings and 

Either * researchers . are not 'affected for most people. 

Although, tTTefttaJ'orl.ty report no effect on 'the reading of professional 

books and Journals, a significant m'inority (2b£ overall and 4M? of 

« 

heavy .system users) report an increase* Communication^ with 
colleagues at, one's own location is more likely to increase than t^ 
degrease. 

Of course, subjective reports' about the frequency of use of varibu 
media are likely to be qijfite unreliable. However, we -did not ask for 
acurate counts, but only 1 f or gross changes: up, down, or about th^ 
iSame. jdShfhall, there is awtendency for the media to add on to other 
modes S^ch^cmels of communication, rather thai! to substitute for 
-tfhem. Previously established scientific and professional networks,- 
, m^ntained by other forms of communication, persist along side^&f the 
new/ .larger, 1 more widespread computer-mediated network. 

• » 1/ * 
Nourished by tills additional communication .in a new ;f orm, various 
'measures of social ties show strengthening. i New ties are- established 
""^n the computer network, and some of the new professional colleagues 
.become personal friends as well as coauthors or 'collaborators. In 
social network terminology, the community becomes not *only larger but 



more densely knit. 

The majority of EIES users' report some general impacts of the system ^ 
"on the way- in which they think , and work. Perspectives are 
broadened-- with exposure to more ideas, theories, scientists, and 
opinions. The second is the subjective recognitiA of the impacts 
caused by the Increased communication with a larger network of 
scientists, 'such as the- ability to gat/" instant feedback" on ideas 
and to "kick ideas around" with -others when a piece of work is in its 
formative' stage. Finally, there is an adjustment in working patterns 
and - habits — one to three hours a day on line fitted into the* 
schedule of heavy users; increases in the speed and ^pace of work; and- 
in fefeli'ngs of information overload and the meed to organize one f s 
work more formally. 

Turning to productivity, as would be expected, the more time spent on 
line,' the more likely users are to subjectively report increases in 
the quantity and quality of *ork accomplished as a result of system 
use. Increases "'in quality of work are more likely to be perceived 
than increases in quanti^t£_(J}*4ycer reports or articles, rather^han 



more) . 



■ ? ■ 

:o r^cprted ii 



Such productivity increase's seem to be linked to r^cprted increases 
in the "stock of ideas" with which to* attack new problems, apd to 
the availability of leads, references and other information which cyan 
be usedj^o'help in one f sT work. 

» * • 

A multivariate analysis, indicates, that "meeting" new people on ligft , 

.plays a central part in the process and outcome of system use. The 

315 



strongest predictor of * subjectively reported Increases in 
productivity is the number of persons met on EIES with whom'fcne 
* subsequently establishes regular exchanges. ,• * 



Perhaps EIES is like an! intellectual lonely minds club or singles 
bar. People come>*-ItJh oping to expand their contacts, establish 
some "meaningful" ctwfiuni cat ions , be stimulated by new ideas. If 
they do meet a lot of intellectually compatible people on line , they 
feel that the experience is a product^e one. 
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CHAPTER EIGHT 

* 

COMPARATIVE MEASURES FOR MACC-TELEMAIL AND RLAfaET 

To- what ' exte,nt are the observations of EIES users generalizable to 
other systems and other types of user groups? A limited amount of 
data are available to\begin to answer this question. 



"Theory Net", a comparable "invisible college" in the area of 

* • 

theoretical computer ^ science, , which used the TELEMAIL 
computer-mediated communication system, was studied using some of the 
same measures as were employed for the scientific communities on 
EIES. In addition, some of the. nysa^sures included in this study are 
replications of indicators tis^d^fcV Robert Johansen and his colleagues 
at the Institute f SfS^^eN^tu^ in their studies of PLANET users. 
Finally, we have one subjective Jreadtion question that was used in 
common for NLS, TELEMAIL,- . ^1ES (See Chapter 2 for a description 
of the NLS study. ) ; 



In this chapter/ the Theory Net group and study will be described. 

r Some of ■ the available data for the Theory Net group, on TELEMAIL will 
be • Meaentedr* Then &e will compare similarities and differences in 
the data for the var^ouar systems. If the indicators used in these 
studies aj-e replicated for a few more types of systems and 
applications^ it 4iay be possible to determine the causes of observed 

^ similarities and differences. ' 
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THE THEORY NET GROUP 



In proposing 'this field trial to tgst the use of a computerized mail 
System, the principal investigator wrote: 

e 

"The tfee^Retlcal computer science research cdmmunity has r a 
well developed grapevine. That is, significant interactions 
between active researchers in the field are already taking 
place over a number of alternative communication channels. 
This gives the opportunity in the proposed test to compare 
the notion of a computerized mailbox and its efficacy with 
' already existing" communication channels. The reason for % 
the interaction within the , community of researchers in 
theoretical computer science are twofold: 

, ,1." The community itself is made up of a .relatively small 
number of active researchers. This means that significant 
interaction is inevitable v since active researchers tend to 

know most of the other active researchers in the area. 

i 

2. Very few research institutions have what would be 
' classified as large' groups of theoreticians. Therefore, if 
* * group • interactions are to take place at all - and* 
inspection of recent technical "articles in the field 
indicates that does take place - active groups ' must 
necessarily interact over " significant distances*" 
(Landwebber 1977:1-2) 



H^ne institutions, "including a NSP representative, were /originally 
included in the • "theory net" group, % With a total /of twelve 
individuals. It was . noted that possible activities, might Include 
"correspondence between research collaborators/ the preparation and 
circulation of x results and reports, the tr|ansmiss*iori\ of results -to 
the *iltor of the newsletter of .the theoretical con^tfcr science 
(SIGACT) community and communication with NSP personnel. " ^The SIGACT 
editor was included in the group, and as time w£nt on, many other 
members were added to the theory group. i ^ , * 
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Self -Reported Characteristics of Theoretical Computer Science 

As a Specialty 

In estimating the age of the specialty, most t*espondents said 10-19 
years* Eighty-eight percent of those responding reported three to 
'ten Journals relevant to the specialty *rea, and all reported npne in 
which descriptions of ongoing research were available* All reported a 
"must atten4^_^early meeting of the theoretical computer ■ science 
specialty. There was also unanimity that there is an "intellectual 
mainstreata" in the specialty, and all of the Theory Net participants 
felt they were "in* the mainstream. Competition is generally 
perceived as moderate and mainly attributed to the high achievment 
drive of some of the members of the specialty area and to competition m 
*~ for funds.* There were no reports of ^s^ongly opposing theoretical 
viewpoints or of unethical competition. 

The picture which emerges is thus of a somewhat more mature specialty 
area than was typical'' of - the research communities on EIES. This is 
reinforced by the reported preference for working in "established 
areas." In terms of basic values, however, th^y lean towards 
emotional committment rather than neutrality, and particularism 
rather than universallsm, just as do the . groups on EIES. The 
scientists themselves are rather young; mdst are under 35* All have 
Ph.D's. They do not write books ih this highly mathematical field, 
but they were working on an average 4 of five journal articles and 
0 almost all published one or^ more articles the preceding year. They 
had spent most orVall of their scientific 'careers in the specialty. . 
-Most consider themselves to* be in the middle to higher /ange in terms 

r - 
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of professional reputation within the specialty. 



As computer 



scientists, all were of course very experienced in the use of 

computers and terminals before using the t^ T ;™lAT T - «y«t:pm j and had 

favorable attitudes toward computers. Howe^dr, on the basis of their 

previous experiences , they tended not to trust computers "for the 

storage* of paperwork used daily. They > anticipated using the system fc * 

only for private messages and reported that they were strongly 

t 

motivated to use the system. 



THE MACC - TELEMAIL SYSTEM 



( 



.v 



The TELEMAIL system (later* named , 8MAIL, when TELENET took over the 

right to use the name TELEMAIL) provides the ability to send items 

such as memos, drafts of working papers, an£ computer program source 

listings or data ,to other "addresses" or "mailboxes." It is resident 
> 

on the Univac computer at the University of Wisconsin, Madison, and 
accessible through the TELENET network. Its design was influenced t?y 
that of other computer-based message systems, particularly -HERMES. It 
has a simple set of cammands that suffice for the beg^aner: 
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EXPLAIN 
STATUS 
PRINT 
TO 

MAIL 
DELETE 
EDIT • 



QUIT 



There are also more complex features, such as the, file system, 
"filters," and a separate EDIT system. (Academic Computer'Center, % 
The 'University of Wisconsin - Madlsoryl 1975, 1977. Updated manuals 
are now available. % These are the versions originally supplied to 
Theory Net members.) The "mail"- metaphor pervades the system, with 
"postmarked" dates and even a "Posj^»Master, " the "mailbox" to which 
questions-" can be sent. Note that In order . to edit, a user must enter 
a separate edit system when the message Is finished, then re-enter 
the MAIL- system when the editing Is done. 

* 

The 1 user interface Includes conventions peculiar to UNIVAC, with the 
use of asterisks, periods, and such to name subfiles. For instance, a 
3ample command is: . « 

COPY S0URCEC0DE»?0RA. PROGRAM to JIMMY=CARTER 

Such a copy command must be used after an edit, before a message can 
be^ sent. As a result, as, we shall see, many Theory Net members 
avoided the editor. 
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The Theory. Net Qroup communicated almost entirely by private 
messages. It was therefore not possible to* observe their behavior or 
to become accepted as a neutral and sometimes helpful observer, as ih 
EIES. An attempt was made to gain rapport by setting up a group 
file*, which could act like a conference. There, a plea was 'made for 
copies of the material being sent among members of the group, so that 
it could b'e analyzed. One person cooperated by. sending -some sample 
zgess^ges; everyone else ignored it, 

n 

"A second source of data, summary monitor statistics on amount and 

type of use, , was. not available * for the MACC system. Accurate 

measures of the dependent variable* had to be abandohed. There are 

only 'subjective estimates from some of the participants on the amount , 

i 

oT time^spenfe on line/ each -week at the time of follow up. m 
* 

Questionnaire data are sparse and incomplete.* This group started 
very small, and was frequently added to. Unlike the arrangement with 

9 

the 'EIES- staff, a copy - of the pre-use* questionnaire was not 

automatically sent to each member as he/she was' added. We therefore 

J. 

have very incomplete "pre-use" data; iti includes only- the original- 
core group, of, members. There were eight responses to the pre-use 
questionnaire, .which was sent out in the early fall of 1978. There 
, was no obvious poi**t at which to send follow-up -questionnaires ; at 
about- the 3-6 month point, when they had been sent for EIES, there 
were plans for ,the Theory Net group membership to ^be greatly 

- -i 
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enlarged, and it thus seemed premature to do a ftollow-up. Thus the 
follow r up^^as actually administered at appoint in time (Spring, 1980, 
at approximately eightee'n months) equivalent^ to, %he post-use 
questionnaires for EIES,' and no comparable post^use measure was 
taken. There were 22 follow-up responses^ f rora the expanded Theory Net 



f group; 



Finally, the study suffered from inadequate contact with the 
principal investigator' and no face .to face contact whatsoever with 
any or the. group members . The EIES groups were clustered .on the East 
Coast, and it was feasy and inexpensive to visit with the principal 
investigators and other key members from time to time, particularly 
since many of them came to Newark So falk to the EIES staff... 
Wisconsin was a long, expensive Journey from Newark. i 

4 

Many of these problems could have been alleviated with an intensive 
investment of time and travel funds; however, such resources were 
simply not available within the modest funding levels for this 
project. 

In looking at comparable data for Theory Net on MACC-TELEMAIL and for 
thevEIES groups, ahy. 'differences or . similarities observed can be 
interpreted, as Supportive of hypotheses, but not as proving or 
'disproving hypotheses. There' are too many differences in the nature 
of the subjects studied, the' systems used, and the timing of the data 
collection, plus poor response for the Theory Net group, to rule out 
many alternative explanations for any similarities -or differences.- No 
statistical tests of jdif f erertces between the two sets of data will be 
made, since the data themselves 'are not fully comparable. 



MACC-TELBMAIL: QUALITATIVE DATA ON 
USES' AND REACTIONS 

Table 8-1 shows the reported uses of MACC-^ELEMAIL by the Theory Net 
group. Generally* the system was used only for private messages. A 
few small- groups . of two or three used files to coauthor papers or to 
coordinate Joint research. ' 

j 

The comments in Table 8-2* indicate some 'general dissatisfaction by 
the computer-sophisticated with the system. The editor ^J^a^-the source* 
of much criticism, and -there are some % complaints that the aiystem Is 
"anachronistic" or not state-of-the-art-as compared to other computer 
systems with which the group members are familiar./ There are also 
complaints about low activity levels. 
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Table 0-1 
Reported Uses of MACC-TELEMAIL 



Note: Ttuese' comments are taken from any or all Na£ the following 
questions : 

— What are the main activities you have been engaging In on the 

system, and with whom? * - - 

* * 

— Are there any ideas that you are using or working with at present 
that you first learned of on the system? 

— Are you working on any projects or papers at the present time that 
have been advanced by your use, of . the system? 

Are you coasrthoring or collaborating closely with any members of 
your group at the present time, using the system? 

— Ar<e there any new uses you have invented for the system that are 
helping you in your work? 

—What tasks, or activities can you suggest 'for your group on the 
system in order to motivate participation? 

* ] • 

1. Short mail messages to coordinate research papers and/or - travel 
with (a few) people. 

Reports "yes" to coauthoring'and working on projects on linfe but 

stives no details. ' 

\ 

3. Uses it for inquiries afcout university policy and activities, 
\recent research, and whereabouts of people, plus "general gossip ■ and 
foolishness. " 

Refereeing, paper preparation and editing, correspondence related / 
tV professional conferences. , Reports as a result of participation/ 
"interest in a 'universal Slanguage 1 for specifying mathematical 
notation in standard ASCII." 

5.. Mail, research document preparation. 

6. Reports research collaboration with one other person (described as 
"a very active research project"). 

/ 

7. "Reading a few system passages and a couple of short letters*" \ 

/ 

8. Mail activities (substituting for phone and U.S. mall). Book 
review column for SIGACT NEWS. Some research with one oth$£ . member, 
exchanging ideas on future Joint work. 



325 

340 



9» Mail and research collaboration with 2 other members. m 

. m 

10. Coordinating SIGACT Symposium Program Committee. Mail to one 
other concerning v research interests. 

11. Simpl^messages, exchange of paper abstracts (2 ot.hers named). 

12. Exchanging brief mail messages. Use \t Tor messages to other 
department members vrhile traveling. 

13* General messages to associates. Items regarding publication of 
SIGACT NEWS. 4 ' 



r 
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table 8-2 « 
Comments or * Suggestions abVit Improving TELEMAIL Features 
or Initiating Desirable New Features 



1. The system is at least ten years out of date. Compared to a 
system like UNIX-(Just an example), the user interface is very poor. 
This is probably because of Univac's OS, but it seems that a really 
usable mall system shpuld be built on a good OS— not just the one 
that happened to be available. 

2. I cannot now use the system until I can have access to a 1200-baud 
dial-up conriection. . 

3. Improve the file handling and text editor. It is < yery difficult 
to use in preparing and sending documents. It is also FAR too 
expensive. 

4. Enhance the ability to write math formulae. 

F ♦ 

(Main Negative 1 Aspects of the TELEMAIL System) 

1. 1 have found' the system unfriendly. 'This- is very disappointing. 
Other "mail systems are far easier to use. 

I refuse to 'edit messages on this system because of the complexity of 
the process. I merely write short messages" and leave the types 
thene. ' - 

| 

I regret not" being able to use an editor and an operating system with 
which I am familiar to, compose text and thenjjimply to send it. 



2. My difficulty is in establishing a regular pattern of use, due to 
the fact that I don't have my, own terminal and, hence., rarely log on, 
I thus do not send # or deceive much mall. 

3. Some peoplAido not check for messages frequently. There is no way 
to "prompt h ^«hem\^ j 

' 4. The system crashes too often. 

5. Bad editor! Bad file handling! 

6. This system needs to be polished in important >ways . 

•7. Supposedly, our group is already heavily into computing. 
Therefore, much of this system is an anachronism. ^ 
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THE PLANET DATA 



m « 



' tv*. 'il^^flikely 500 'members* of more tha^H^>?cganizations were 
9 aI_ observed, • using PLANET (or, in a few" ffases, the related FOrfUM 
■.system)/ by Vallee * et al. as part of a project conducted by the 
Institute $o> the Future.. ^Among the organizations were NASA,* the 
{J/S. 'Geological Survey, ERDA, 1 and the Kettering ^Foundation, The 
.conferences* lasted from 1 week to 2# months.. , Applications included 

t r 

topical conferences 6n food. and climate, individually guided 

£duoa£ii^ ' t^phnology transfer, and psychic research, as well as the 

management and " coordination „ of ^techni^l projects or Joint report 

writing (Valiee et al., 197b:xv^), 1BJB of the participants responded 

to a post-use queationnare . These tended to txe the heavier users of 

the systems, with 40X of the respondents above the highest - quintil 

in terms of Clumber of sessions and another 30% .between the highest 

and/ the second quintile/( ibid: 109) • We thus have a very wide range 

of s^zes and types of groups and -applications, plus an 

* unrepresentative set of survey responses/ 1 Nonetheless, comparative 

responses for 'the same post-use questions included in the'EIES and 

. , MACC TELEMAIL field trials may be informative. f 
' t 
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SUBJECTIVE ftATING SCALES 

/ Tabia" b-3 show? the comparable means for overall subjective 
satisfaction ratings for EIES and TELEMAIL . - There are many 
similarities, such 'as almost exactly the same average for the overall 

* * * 

rating of the systems as "extremely good" to "extremely bad." The 
means are exactly the same for "easy" to "difficult^" However, there 
' "are also some .interesting differences. EIES is seen as more 
friendly, .more' stimulating, and more *Tun. .- f^f ever', it is also seen 
as more time wasting and demanding, probably because of the much, 
larger volume of activities on line. ( * f 

DACOM Scales 

# TheMCOM ratingg ° f the extent " to which MACC-TELEMAIL, EIES, and 
PLANET were sat^fj^ory f*or specific communications functions (Table 
. ' ...8-4) yielded similar results for most items, w^ph.the exception of 
"getting to know someone." For all three systems, giving or 
^ , receiving information and exchanging opinions were the tasks_for 
which ££e highest degree of satisfaction was reported; bargatwfpg 
*|£jind persuasion were among the least satisfactory for 
^""computer-mediated communication. Using a criterion of more than a 
(Jkirtt's deference between means, the only clear difference is in 
"getting to know someone-;" . for this, EIES received higher ratings. 
This can probably be attributed to 'differences in design, such' as the 
V j>rfesence of a public directory in EIES and the group vs. individual 
orientation » of conferences " as compared^to messaging. * Another 
' apparent difference is that the TELEMAIL group does not seem to have 
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as' much difficulty with resolving {iisagreements on line. This is 
probably because of a combination of* 'the fact that group debates 'are 
/hardly ever held v'ia messages, as compared to conferences, which are 
often set up -specifically to find and discuss differences of 
scientific opinion;* and because, the Theory Net group studied 
utilizing TEL EMAIL within theoretical computer science .is not in a 
scientific community that is undergoing. a lot of disagreements. 

Experiences Communicating via EIES; TELEMAIL, and PLANET 

Using Ltfems. originally designed by the Institute for the Future for 
evaluation of PLANET, we can get another set of comparable measures, 
thisX^me for the three systetos (shown in Tkble 8-6). Most of thef * 
averages ire very close. Users of all three systems tend to 
"sometimes" feel di-stracted by the mechanics, tO'> T, sometimes" feel 
constrained, "almost always" able to express, their views, and, 
somewhere in the "Sometimes" to "almost always" range, able to get an 
impressipn of personal contact with others. The only difference is 
that the users, of the mail system less frequently feel overloaded 
witji information than do the users of the two conferencing systems, 
who sometimes find a large number of items waiting for them in a 
large group conference. ' \ * 

System as Useless to Revolutionary: TELEMAIL, EIES, and NLS 

' An it^m desired by Edwards for her NLS evaluation was used for the 

EIES, and TELEMAIL studies in order to obtain comparable measures of 

feelings about the usefulness of the .systems and the extent to Which 

» 

they were potehtially "revolutionary" (see Table 8-6). Remembering 



that dissatisfied or lotf-level users were least likely to complete 
the questionnaires, it is npt surprising that, for all systems, 
responses are generally more posit % ive-tar£r^4^fe '""neutral" point and 
that,u*ers are likely to feel that their systenNhas at least "certain 

b worthwhile uses," The only clear' difference seems to be ' In the 
extent to which usjers feel that the system is "revJlutiQnizing" 'their 
wortc N and ' communications This is not at all as frequent an 
evaluation for the simple mail system as for the more complex systems 

} designed to support a wider variety 'of- communications and work 
functions. 



ERIC 



331 



34 C 



• ♦ 



Table b-3 

Overall Reactions' to MACC-TELEMAIL and EIES 



Means (7-point 


scales; 1» 


Highest 


I teen 


TELEMAIL 


EIES 


Extremely Rood— extremely bad 


2.9 


2.b ' 


Stimulating-boring 


3.9 


2.5 


P roduc t i ve-unnrodur t i 


^ . 1 




Great fun-unpleasant work 


3.9 


2.7 


Time saving-time wasting 


2.3 ' 


3.7 


Not f rustrating-f rustrating 


3.9 


3.9 


Friendly-impersonal 


' 3-9 


• 2.7 


Easy-difficult 
Not* demanding or 


-2.9 


2.9 


1.4 


3.4 


intrusive-very demanding or 






intrusive ' 







Sources: MACC-TELEMAIL Follow-Up Questionnaires 22) 
EIES Pollow-Up Questionnaires (N - 111) 
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Table. ^4 : 

DACOM Scale Measles - MACC-TELEMAlL, EIES^ and PLANET f 



Means 



Function 


TELEMAIL 


EIES 


PLANET 


Qlving or receiving 


2.0 


2.4 


2.1 


information 








Problem solving 


4.0 


3-9 


3.4 


Bargaining 


4.4 


4.1* 


4.2 


Generating Ideas 


3.8* 


2.8 


2.6 


Persuasion 


\ 4.3' 


4.2 


4.6 


Resolving disagreements 


. 3.5 


4. i 


i.i 


Getting to know someone 


4. a 


3.3 




Giving or receiving orders 


3.2 ' 


3-2 


/.4 


Exchanging opinions 


' "1.9 


2.3 


/2.1 


Sources: Theory Net Follow-Up Questionnaires (approximately 
N - 22) * . 

EIES Post-Use Questionnaires (approximately 18 months; 



ly lo, mo 
j N » JLO 



02) 

PLANET: Qomputed means to 1 nearest .1 from raw data included on p. 183 
>of Vallee et al., 1978. Scale reversal used 'to obtain comparable 
values. 

Questions: How satisfactory, do you think the system Is for the 
•following activities? (1 - completely satisfactory, 7 3 completely 
unsatisfactory) * « 
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Table 8-5 

Experiences Conununlcatlng via TELEMAIL, EIES, and PLANET 



Peeling 

• * 

Distracted by the mechanics 
of the system* / 

Constrained in the types-of 
contributions you could make 

Overloaded with information 

Able to express your views y 

Able to get an impression of 
personal contact with other 
participants 



Means 
TELEMAIL 



EIES 



PLANET 



3.3 


3.1' 


3.2 


3.1 


3.2 


3.6 


4.1 


3.0 


3*. 6 


2.1 


2.1 


2.0 


2°. 2 


2.6 


2.-6 



Source: EJES Pollow-Up Questionnaire, N », 110 
TELEMAIL Pollow-Up Questionnaire, N = 22 

PLANET means computed from raw data reported on p. 182 of Vallee et 
ai., 1978. 

Question: Thinking back over your experiences with the system, how 
frequently have you felf... ("always ■ 1, "almost always" ■ 2, 

"sometimes" ■ 3, "almost never" ■ 4, "never" 3 5) 
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Table 8-6 

^Overall Ratings -of Systems as Useless to Revolutionary: 

EIES, TELEMAIL, and MLS 



I think It Is useless and 
should be discontinued. 

I think It has its uses for 
others, but not for me. 

I am skeptical but am giving 
It a try. 

I am basically Indifferent or 
neutral. 

I think that It has certain 
worthwhile uses for me. 

I think It Is very useful In 
many respects. 

I think It is revolutionizing 
my work/communications 
processes. 

Total 
M 



EIES TELEMAIL 

0 0 

81 , 0 



m% 
Zi% 

17% 



100% 
107 



100% 
19 



MLS 
1% 



1% 

5% 
2% 
22% 

23* 



1002 
94 



Question: Which statement bjpt describes your present reaction to 
Sources : 

EIES: Follow-Up Questionnaire- 
TELEMAIL: Follow-Up Questionnaire 

« * 

MLS: Post-Use Questionnaire (Edwards, 1977, p. 105) 
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-SUMMARY 

Much early work In anthropology fell Into the category of 

"ethnography": ®the description of a single society. Later, as this 

« 

descriptive material accumulated, "ethnology," • or the comparison of 
similar institutions across societies; becatoe possible'. A priority 
for future research on computer-mediated communication systems should 
be sufficient standardization of the types of data collected and the 
measurement Instruments used so fchat % an "ethnology" of, 
computer-mediated social systems becomes possible. 

Based -orP^the limited compa>\at>ve* data available, there is a great- 
deal of similarity In user ratings of the characteristics of the four 
systems covered in this chapter (EIES, PLANET, MLS, MACC-TELEMAIL) , 
despite many differences in system design. The main* difference seems 
to be between the simple message system ( MACC-TELEMA IL ) and the more 
complex systems. The simple mail system is less "friendly," less 
fun, less stimulating, le ®^ useful for "generating ideas , w and 
overall, lfeTI~^evolutionary" in its impacts on users. On the other 
hand, it also takes much less of its users 1 time, is felt to be less 

0 

Remanding and intrusive on them, and less likely to overload them 

with information. 

« » 

\ 

i 

\ * 
j 



— * 



CHAPTER NINE 



SUMMARY AND CONCLUSION > ." 

f ' . 

—If you were rjading this report on your terminal, you could at this 

point loop back to review the highlights of* each, chapter, skimming 

V ' • 

concise topic descriptions and deciding if you would like to read a 

fuller summary of each point . With a linear text, this is not 

possible. The , closest equivalent is to* simply raise the main points 

which have been mac 




) - • . 

1) Methodological Problems 



a) The design of the study postulated a fixed group of. 
scientists using a specific computerized conferencing system for 
a period of eighteen to twenty-four months, with objective 
behaviorial and subjective attitudinal data collected at several 

9 

points in time. In reality, a cortstant turnover in group 
membership occurred. This combined with steady changes in the 
nature of the system, and non-response on questionnaires to give 
us incomplete data for most participants. As a result, for 
analytical purposes, the data can be treated for the most part 
only as several cross-sectional surveys ^ rather than as 
longitudinal panel data which is more amenable to causal 
hypotheses. m 

b) Unknown Oenerallzablllty 

\ ' 

The scientific communities were not representative of all 
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scientific research communities, but tended to be fairly new and 

« 

cross-disciplinary areas. In addition, there was considerable 
self -selection within the communities, both in regard to initial 
invi tat ions/agreement to participate and in amount of actual use 
made of the system. Finally, we do not know to what extent the 
scientists are % similar to other professional and technical 
^people/ * 1 

2. The strongest predictors of acceptance of a computerized 
mpr\Z erenclng system are attltudlnal and motivational variables 

♦ 3 

rather -than any "objective" characteristics of users, 3uch as 
previous computer experience or typing, ability. Such variables 
Include expectations _^baut how useful the system will be and how 
many people one knows who will be on llr\^. 

s 



With self-activated learning, as occurred with EIES, those with 
poor Initial expectations of the usefulness and Importance of 
- ^ communication with . others via the computer system are likely to % 
never -sign on at all or to lack the motivation to remain through 
the learning period. The very high drop-out rate among invited 
users is a serious problem for the future of computer mediated 
communication systems. 

^ ' J 

3* User group is an Important contextual variable. The same 

3ystem is likely to be perceived as having good or bad software 

» 

feature^ and as being a productive or an unproductive means of 

working with others, as a function of group membership. Oroup 

membership includes such variables ^s whether or not there is 
& effective leadership and the nature^pf the task the group is 



wortaing on. 

4. There is a process of "evolution" in user behavior, whereby 
t more experienced users change their .preferred ' mode of 

interaction .from passive menu selection to active 'Command 
definition, expand and ^rhange the nature and number of feature* 
in \ tf i ijjjjjuli MiitJ IJTt i i nf i' i mi system whicfh they . consider 

necessary and usSfuL, and expand the range of communications 
, functions for whJch the medium is seen as satisfactory. 

5. Those, who do get through the learning period and actually 
.participate in group communications tend to rate the syttem 

positively in terms of such characteristics as -being easy, fun, 
and productive. They also tend to endorse\ specific -design 
choices that • were made in the EIES system/ such as forced 
delivery of private messages (inability reject them before* 

they are ever printed — out-) — -«id--4L progression of m levela of 

^ ----- 

• interfaces whereby users begin with m§nus. The strongest 
predictor of subjective satisfaction with the system is. the 
extent to which it has expanded social networks through 
facilitating "meeting" and working with new colleagues who share 
one's interests. 1 

6. The scientific communities tended to report ttfat there were 
as a result of uae some ^clarifications of theoretical 
controversies in the field, a^ increase in total communication^ 

\ within the specialty, and an increase in contacts across 
disciplines or specialties . There were no decreases in 
communication with off-line colleagues as a result of systerrr 

339 



use. 

7. Subjectively perceive^ increases in "qUality" of work as a 
result of- system use are more frequently reported than increases 
in 'quantity of work performed. Exposure to a broader range of 
information and ~ ideas than otherwise possible, and the 
availability of a <much larger network of people who may be 
helpfyii . whtfn one doe-s want information and assistance with a 
specific project are among' the benefits that are seen as 
increasing productivity. In terms of media substitution, there 
is soafe decyrSase - in telephone and mail use as a result of 
substituting computerized conferencing, but. no decrease in 
travel or in reading of professional books and Journals. 

•8. Even though there were marfy 'differences among 'groups, 
applications, and systems, results for several^ measures of 
subjective * satisfaction replicated on EIES, MACC-TELEMAIL*, 
PLANET, and NL3 are for Jfche most parl^ very similar. This 
implies that there are some general characteristics of all 
computer mediated communication systems in terms of user 
reactions*, • . * 

Tentative Conclusions 



The above points are simply a, review of conclusions for which data 

* 

have been presented iln # the proceeding chapters. Comparing the 
experiences and the degree of success of, the various EIES groups, *he 
following- kind of tentative conclusions emerge: . 
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^1. Learning to use a new medium of communication and to 
% effectively integrate it into one's work* patterns 1^ .no simple, 
' easy matter. Although EIES members could learn the basic 
mechanics of usir«. the system In a few hours, they did not 
became fully comfortable wtth it and able to utilize some of its 
0 potentially most useful features, such as Joint document 
~~ production, until fifty to^one hundred hours of experience. 

. > 

2. If a user- "group does not have offe, or more persons willing to 
take the^responsibility for an active leadership role, spending 



on. the average an hour or more a day on line to organise and 

*' * " » \ 

^'"stimulate the interaction and task coordination, an applicatiq 
is likely to beia failure. 

3. Participants should feel that the task or activity in which 
they are engaging on line is important enough so they are 
willing : to MAKE TIME to spend at least an hour a week on' line. . 
Less regular . participation leads to frustration for group 
members when messages ^are not picked up and responded to, and 
for the user constantly forgetting how to use the system and 
never becoming proficient and comfortable with it. * 

• 5. Groups like to be able to Jointly develop some special 
structures pr commands to help them in their particular tasks. 
This theme will be treated more fully in a subsequent report. 

'* 6. There were no dramatic "'scientific revolutions ff > in the sense . 

■ \ ■ , - ' 

of new paradigms emerging during the eighteen to twenty-four 

i 

months of observation. However, there was prtogress towards the 
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clarification of theoretical controversies in most groups. Most 
importantly^ ther^was an expansion oCthe professional' network 
wi^th which active 'members of, the system regularly interact; and 
a .feeling of greater awareness of varieties of work in tthe area 
and of the. a\?*£l£bllJ|fcy of new "sources antf- types of Jjjf ormatlon 
useful- in scientific work, through this network^ 



Looking at the high drop out rate contrasted tfith the; testlm onlays of 
the confinned users, one wo^^rs if p^rh^s (?C is like religion: it 
only helps' if you have faith th£>^lt 1 
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&§£NINQ AND OBJECTIVITY: PARTICIPANT OBSERVATION -AND THE RAIN DANCE 

Initially, this study was designed to measure^ome rather limited,, 

pre-defined . impacts of a computerized conferencing system^ on the 

communication patterns, paradigms, and productivity of scientific 

research communities. The process . of 'communication via computer was 

seen as merely an intervening variable. As the study progressed and 

the communication patterns were observed, however, 1 the process of 
V 

communication via computer .emerged as a phenomenon worthy of study 
" and description in its own right. Moreover, such description is 
likely to be generalizable to most professional and technical users 
of the medium. The study thus shifted in fodus as it progressed, 
'f 4 pora the sociology • of science to ,the sociology of individual, and 
group processes in adapting to a new communications medium-. * In 
addition, it became obvious that this new form o^ communication "had 
3ome perhaps unantielpated consequences for the* participant s . Even 
though some of the scientists wondered if the aihount of time they 
invested in such ' communication in fact had any direct productivity 
pa^offls, they c<£itinued to participate. \ At the sajne time, the 
detached observer became somewhat caught up in a shared belief by the 
members of the communities that their activities • hadT some importance * 
and significance for the future of scientific research, and. society 
as .a whole, even if. they were not -quite 'sure how to explain the . ) 
significance. - 

There are parallels with the cautionary tales of Paul La^arsfeld as 
he discussed the dangers* of ; f going \native* f while studying the rain 



dance, anc3 with the penetrating functional analysis of Robert Merton 



i. 
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in illustrating the concept of latent functions with that very same 
ritual gathering* * Lazarsfeld cautioned his students somewhat as 



follows: 



I 



You can observe th^^rain dance and * maintain your 
objectivity. You car^e*ven participate in the rain 
dance, *^Tnd gain a subjective understanding of its 
meaning for participants. But when you start to 

believe in the rain^'daihce when you start scanning 

the horizon anxiously for sight of those dark clouds 
signalling that your activity is indeed going to bring 
rain - then you are in trouble. You h§ye gone native. 



As this study progressed, I did begin to share the belief of many of 

the participants that their experiences with a new technology had 

important consequences # for the ?ut^re~ of "not only scientific. 

communities, but also human society in- general. This belief could 

not be substantiated with* any objective evidence of productivity 

gains. There are only the subjective feelings of the most actiye 

participants 'that their electronic tribal gathering was beneficial; 

that if nothing else, they enjoyed it and were stimulated by it. 
• < 

Prom the point of view of traditional functional analysis, the 
premise "is that if a g^oup or society persists/ in a pattern of 
beivavior, then it must have some beneficial outcomes. These outcomes 
may be "latent functions" - neither intended nor necessarily 
recognized by the ^participants, as 'contrasted to the "manifest 

J 

function©, " those publicly announced, of ficially endorsed goal3 fop 

1 

the activity (Merton,' 1968, p. 1195. In the case of tne rain dance, 

Jo 0 ■ W 



there is no objective evidence that, the amount or form of dancing 
affects the probability of rainfall, ^ parallel that struck me is 
that an important scientific breakthrough may N be Just as 
unpredictable as. rain-: hard work and trying alone may -not" cause £t 
to happen; other enviroijnental variables are a controlling factor. 
However, in the tradition of Durkheim (in Tire Elementary Forms of the 
Religious Life), the rain daace may have ImpOftant functions for the 
social ^solidarity of the^group. It- brings the members of* the tribe 
together, gives them a chance to reaffirm and clarify their shared 
beliefs; and creates the opportunity for new marriages and alliances. 
Such -new alliance^ may indeed help the participants to better cope 
wiph their environment and engage in fruitful cooperative efforts, 
over the long .run* * 



For. the r^ew class of computer-based communications systems, of which 

EISS is a forerunner prototype, but only a single example, .it is hard 

to quantify tftose latent functions, which may be the most important 

in the long qun. -In retrospect, a period of eighteen to twenty-four 

months seems too short, to expect to see large increases in scientific 

productivity of individual participants or dramatic paradigm shifts 

in the user community as a whole. EIES activity was, after all, only 

a •small portion of their total professional lives -and activities. 

For thfe 'relatively "h^avy" use-rs who spent 100 or^ore- hours on line 

over eighteen^to twenfry-four months, this is still only a few hours a 

weefcT-^The most important consequences sesm to this observer to be 

the enlarged and strengthened professional network of colleagues, the 

greater understanding of the work of others and how it relates to 

one's own; and the feelings of* having been exposed to a wide variety 

of information and ideas that wobld form a permanent resource that 

* ♦ 
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might be utilized -during the rest of one's professional life. 

An emergent objective of this Vtudy thus became to describe and 
understand the communication activity itself, its forms and 
variations, and the feelings of the participants about it. , As in an 
ethnographic study, one can gather some qualitative descriptive data 
by observing and participating, and ' some more quantitative 
descriptive data by surveys of -attitudes and census counts of 

\ 

activi^v /patterns . When we have a number of such descriptions * for 
varioua/coWputer-based systems and a variety of 4 user communities, we 
/ will' be in a better position to try to prove "cause" and * f effect." 
In the meantime, the forms and rituals of communication via computer 
are at least as interesting to study as the rain dance, and. 
, potentially much more important for the future of 'a society which may 
be forced' to choos* cbeap telecommunication alternatives in an era of 
scarce resources. Stretching the analogy between studying the rain 
dance and studying computerMne^i^tted 'communications to 'its fullest, 
the rain dance may be # seen as a cultural reaction to a crisis 
situation with whicji the society does not know how to deal; so may 
the large-scale experimentation with new computer and communication 
technologies. 
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APPENDIX A < 

Pil-USE QUESTIONNAIRE V 
Study of the Impact of Computerized Conferencing 
Upon Research Communities 
(Copyright, 1977, Starr Roxanns Hlltz) 

Tour cooperation In completing the following questionnaire, before you 
participate In the system for more^ctlan an hour or so, is vitally necessary for 
a thorough and proper evaluation. The questions are designed to collect some 
information on your general background, your communication skills and styl£, 
your access to the conferencing system and your predisposition concerning its 
us*. You should be able to complete the answers in about '30 minutes. 



Directions . ^ 

Most of the questions are structured so that they require only a check or 
a simple numeric response* Som^, however, request you to list or describe items* 
Plsase type or print your response as clearly as possible. Where you do not 
know or cannot make a rough estimate of the answer you may leave it blank. 

Kotice that a continuation page has been attached to the end of the question 
nalre should, you need additional space to answer or clarify your response to any 
of these questions* ^ 

Your Name - 



EIJS Group Name/# 
Job Title 



J Your Employer ^ 

City a State 

M 

This questionnaire is voluntary and • in no way conditions youp participa- 
tion in the system. U you have, fot some reason, an obgnctlon to filling out 
this questionnaire, please note your objection below an^retura it to us. Or, 
if the case applies,- not* your objection to any single question and leave it 
blank. 



Object ion r 



A2" . .360' 




;-USE QUESTIONNAIRE 



DATE FILLED IN 



CODED ID ONLY 



# (Cover page will be removed tQ 
preserve confidentiality) 



TURN PAGE TO BEGIN 



^ — J 



3; 
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At 



»«rt I. W EIBS Group V*«a*arcfc Specialty (Group # / 
■ ^ ElErVf?* * °»« Ascription in your oW words of' fat 



4 



Employer = 



Academic * 71 

Gpyt. , , • 4 

Priv* Research * 9' 

Business 2 

Medical 3 



, ^ ' 89 
1 missing ' . * ^ ' #• 



• * 

8 • 5 * s a ^atate and distinct research axea? 

■ .' - . ':• .';«;s:r.^ *~ iJ « >~ b «° -?^» 5 ««.- ^ : 

• • ' a, 

v*009 ' ^ 1 - = - ' 20 5-9 = 35 M+y**"'' .TV>t-8« t N 

4. What is the total number of journals in which' articles relevant m 
your specialty area are likily to appear? arclcies "levant to 

. * • ' A 

. — : none ($\ 7 * n ' 

• "i -i— ritT % f - ?oV M - 

- (4) . IS . 11 - to » Kn 1 .-Off or more 

"\ ■ Tot « Ans. 85 

% • 5 " iL^Tf^l J 0 "" 31 ° r neW8le ""' or other published Wee in which 

. ; < ' " ♦ 

. • „ . \- <1>-2Z_ No f * 

• ' Tbt. Ms. ' **, ' . 

^S4 Tfes: please liat: 81 . 



I 



ERIC 




6. U there any" on. *L.f or convention which * u *° 

order to. keep up. with mutch la your specialty? (IP yea, plus* / . 

. list*, i. * - J 

m .7 no (2) < — lek-^ .\ ' 



7. Could you Hat the four major outstanding people In your entire 
specialty an£ the extent to'whicti you know them personally and/or 
ere in direct contact with, them? 



' / 
/ 









t 






Extent oL.Current Contact 


/ 

Rarely , 


i 




\ 


On EIES 


Not 


on Tot. 


Constant 


Frequently 


Occasionally 


Never ' 

< 




*• 




— 4i- 


_-70 


1 = 12 


2 = 21 


•3='22 1 


4 =14 


5 =6 




b. 


24 




70 


1 =113 


2 = 12 


3 = 22 


4 = 18 


5=9 




c- 


?S * 


40- 




1= 6 


2= 13 


3= 23 


4* 21 


5= 7 


» ^ 


d. 


—20 — s 


36 


65 , 

56 


1= 4 


2= 6* 


3= 26 


4=11 

\ 


5= 12 



75 



8. %0 many members of your EIES User Qroup do tou know either profes- 
sionally or personally? Tot. 213 

1-5 = 153. 6-10 "= 27 < 11-20 = 19 21-79 = 9 All = 3 > Most = 2 

9. - Is there a commonly accepted "intellectual- mainstream" "In your, specialty? 

(1) 36 * Yes (21 49 N o Tot. = 85 . . 

lOi If yes; to what extent do you feel that you and those with whom you 
collaborate^are in the recognized intellectual "ma ins t^r earn of your 
specialty, or conversely feel you -are "isolated" or "peripheral ? 
(circle 'one) 

.Neither in 

Completely in Somewhat in ' the Mains cream Somewhat Completely 
the Mainstream the Mainstream nor Isolated Isolated Isolated 

i 2 ' * ' ■ '■ r 5 

15 .14 . 13 \ . 6 " Tot. = 48 

11. How would you rate the degree or intensity of competition within your 
research Specialty ? ^ <J r^code' 

4 * Very Intense . Moderate- Low \ Nonexistent 

Intense «r ^ . • • 

1 *2 3 * 5 

Tot. = 62 

17 • . • 42 23 



" ' © " ' A5 
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12. What 

yes.- 34 
21 

38 

7 

21 
11 



«r. the r.„on. for.** competition? '(Check'all that apply^ 

-No=_|9 Scarcity of' or competition for funds 

W Rival groups of collaborators 

High achievement or success drive 
«of persons in the field * 



Tot. 
73 



_|$Soaa persona act unethically 

Strongly oppoaing views 
_g2 Other (please describe) : 



Name 



Spec i 



Nnno = |4 



bther 
or 

Esau 



-Sp«q, "2 



_19_ 



-Spar" 3 



Spec. Importance (Scale 0-6) 



0 




7 


1 




6 


2 




15 


* 




11 


4 




22 


5 




14 


6 




13 






sr 






Mcfre 


Less 






time 


time 


Tot. 


4 


41 




= 89 


2 






_= _ 










4 


•• 7 











> 



V 



9 
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During an average week, approximately how many hours do you spend on 

Zch ot the following kind, of activities? (First Wthe total for all 

Professional activities ^then the number of these rented only to activities 
vijjain your specialty area)* 



Total 



n^r^t re* »«T gh ^ctflvltles 



Wr^lntt pa pers, books, etc* 



Education 
teaching 



i fl >min ff ; reading bo oks or journals 



attending meetings, 
seminars, etc. 



Administrative and support 
aStivit^es ( committee 

nee/ings, memos « etc*) 

Telephone * » 

Jnalde vour or ganization 



outside vour organiza tion \ J ^ 
nonsuiting 



funding (grants applications 
or other resource acquisition 
activities) 



Other prtofessional activities 
(lipase specify) ! 



Total. 



lours in Specialty , 

only % spec. Imp. 



<6 = 12' 
"6-10 = 4 



•11-19 = 6 



-20-49 = 29 



50+ = 31 
* 82 



.7 



2. 



Please list the names' of any persons with whom you have co-authored or 
collaborated in research curing the last year, or during the current one 



0 = 16 

1 = 9 

2 = 11 



3 = 16 
4-9 = 31 
10+ = 4 



Tot. 87 



whit is the total 



Considering all current personal communication modes, 

number of different' individuals within your research specialty with whom 



4. 
5. 



,* 20-49 = 15 
Q = 9 



50+ = 17 



you are currently in contact? .0=3 

1-2 =9 3-5 = 11 6-9=6 10-19 = 25 

How many of these arm- in your EIES user group?_ 
^-2 = 24 3- 5=22 ' 6-9 = 18 10+ = 16 Tot. 81 

Scientists are sometimes anticipated by others in the presentation of-\ 
research findings. That is, after they t have started work on a problem j 
another scientist publishes its solu'tioh. How often has this happecedf^p 
you in your career? (Please exclude cases where a solution to your 
problem was published before you statted your own work. Circle one.) 




Recodcd 
Constantly 

1 • 



6. How concerned are yea tha 




.me 



Cons t«nt 




lUrely Never 
4 5 
61^ 31 .28 
Eg anticip ated in your ^urrent work? 



Sequent ly 



to Time 
3 * 



Rarely 
4 



58 



Never 
5 

23 



Tot. 90 



Tot. 90 
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Ceneral Principles of 



:iples 
idVfeeJ. 



Science 



D«»crib€d\bftlfl<r are^tvo sets af conflicting general principles which cftf 
guide the conduct and evaluation of scientific research* Please read each 
set of principles with your specialty area in mind* 

Principle A. Emotional Neutrality ^ 

Scientists must be emotionally neutral and impartial towards their 
ideas If they are to stand a fair cfiance of ultimately being proved valid. 
Conducting an investigation with anything less th|n an impartial frame of 
mind possesses the danger that the scientist will bias results and be 
unable to give up hypotheses when they are indeed false. 

Principle B. Emotional Commitment 

Scientists must be emotionally committed to their ideas if they are to 
stand a fair chance of ultimately being proved valid. Unless a scientist 
believes intensely in his or her own ideas and does everything legitimately 
in his power to verify them, there is the danger that he will give up his ideas 
too quickly. Initial inconclusive signs of negative evidence do not warrant a 
reorientation of research efforts. ' The scientist must believe in himself 
and his own findings with great conviction. * 

A , ^' • * . 

7. 0» the basis of yoti»^QWB_expterience and observations, to what extent does 
each of the' principles tend to govern the everyday .working behavior of * 
most scientists in your specialty? (Please circle one ifufeber). 



v fccoded 
Tot. 85* 



A 

t Signif- 
' icantly 
More 
Tfiaa B 



A 

Moder- 
ately 
More 
Than B 



Both 
Tend to 
Govern 
Equally 



i 

Moder- 
ately 
More 
Than A 



B 

Signify % 
icantly 
'More : 
Than A ' 



Neither 
Tends to 
Govern 



8. 



Recoded 



Tot. 89 



•18 

To what extent does each of 
everyday working behavior? 



A 

Signif- 
icantly 
More 
Than B * 

1 



7 A 

Moder- 
ately 
More 
Than B 



14 ^ 53 ^ 

these prin^ples tend 



Both 
Tend to 
Govern 
Equally 

>■ 



to govern 



own 



24 



17 



B 

Moder- 
ately 
More 
Than A 

4 

19 



B 

Signif- 
icantly 
f More ' 
' Than A 

5 

m 

26 



Neither 
Tends to 
Govern 

6 
3 



~ 7* — 

v. ■ . 



9. fo what- extent do you believ* Chat each of th. principles ojWTto 
iovern thebehayior of scientists in your specialty. J 



Recoded 



A 

Signif- 
icantly 

Ho re 
Than B 



, A 
Moder- 
ately 

Mora 
Than B 



27 





B 


B 


Both 


Moder- 


Signif- 


Equally 


ately 


icantly 


More 


More 




Than A 


Than A 


3 


' 4 


5 


23 


20 ' 


14 



Neither 

Should 

Govern 

6 
3 



Ci The Irrelevancy of Personal Attributed 



27 

Principle 

4 

The personal attributes of a scientist are completely irrelevant in 
Judging results and claim- to fc knowledge. Each claim in science is judged 
impartially on its own merits by' its ability to starid- up to rational, empiri- 
cal test procedures without reference to the particular scientist. 

PrinAple D: The Relevancy of Pe rsonal«Attributes 

The personal attributes of a scientist are highly relevant in judging 
results and claims to knowledge. In reality the w^k of, some scientists 
' is given credence over that of others. It is necessary to know the personal 
characteristics, -background and motivations of a scientist before one can 
prdperly evaluate his or her work. . / 

As above, we wish you to indicate the extent to' which these two princi- 
ples tend to govern the everyday working behavior of most scientists in your 
specialty ; tend to govern vour own everyday working behavior, and ought to 
govern the behavior of . scientists in your -specialty. 



'( 



Recoded 



Tot. 
85 



89 



88 



10 \ 




c 

Moder- 
ately 
More 
Than D 



. Moat scientists 

/ 1 



11. 



12. 



Tour own 
behavior 

i 

i 

Ought to 
.govern 

1 

. 27 



r 



27 



Both 
Equally 



3 

11 



D ' l\ 
Moder- 
ately 
More 
Than C 



47 



D 

Signif-" 
icantly 
More 
Than C 



Neither 



2 

22 
2 

20 



3 

12 



40 



21 



6 
1 



37b 
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\ Part III Background Itema (Pleaae attach a vita . If available; and 
; omit items covered in the vlta )« 

. 1. What ia your age? 



88 

87 

76 



2. Sex: 



(1) 
(2) 
(3) 

(1) 



2 


under 


25 


* <4f 


Ifi 


45 - 


54 


28 


25 - 


34 


(5) 


3 — 


55 - 


64 


38 


• 35 t 


44 


(6) 


1 — 


65 & 


over 


5 


feme J 




(2) 


-83 


male 





3* Please list your academic degrees (Degree, Subject, Institution, and 



year)* 

v ■ 



Bach = 2 
Bach = 5 



Masters = 10 
Phi D, MD =59 



81 



No Degree = 1 

yrs = 16 
5-8 = 29 
10-19 = 18 
■ 20+ = 9 

4. Have you ever won a prize, special award, or been elected / 
to an honorary scientific society for your research accomplishments? 



.81 



(l> 46 no 

(2) 35 yes (Please list) 
» 



7 J 

5* Prq/^ssttmal Publications (please try to give ex*c? numbers published in 
6V>^eir or underway; estimates are fine for previous works.) 



Currently in 
Progress 



Text bibks 



Other books 



Journal articles 



Papers presented 



Other (describe) 



Published in 
Last Year 



Published or presented 
during 

Total Professional Career 



Tot. 73 



"90 



6. I am more interested in generating a large number of alternate 
explanation* for any problem than in .pursuing one exclusively 
in detail. • 

* Strongly Neither Agree Strongly 

Agree Agree nor Disagree Disagree Disagree 

1 '2 3 . . 4 5 



0 

ERIC 



14 



22 



30 
A10 



20 
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ot. 

n 



0 1 

7. ^^ prefer to work in well-established research areas* 

Strongly Heither Agree Strongly 

. Aaree Agree nor Disagree Disagree Disagree 

1. " 2 * > 3 4 5 

2 31 49 10 

8. Bow well known is your work, within your specialty area? 

: 2 : 3 ; * 4 : 5 : 6 : 



87 



1 

Practically 
unknown > 
A 



14 - 
Coefeents: 



Average 



15 



16 



Ranked 
at top 

of Field 



13 



s 




i 




9* Do you think that the EIZS system will affect familiarity with or the assessment 
of your work?* Explain* v * 



75 



Yes, Considerably = 2? 
Yes < = 26 

Maybe 
No = 12 



\ 
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All 



47, 



IV Comunicatlon Skills and Facilities 
Is English your prAry language? 

t 

" m US Yas (skip to question 2) 

' m) 7 Ho . 

If not, what Is your first language? 



If English Is not your first language, do you Consider your English 
to be on a par with your primary language as to; * 

Writing (D_7 Yes (2) 2_ Ho 

Speaking (1) Yes (2) _ Ho 

Reading (1)^8 Yes (2)__2^_ Ho 

Bow would you describe your English reading speed? 

- i. 



(1) 17 Vmry fast 

(2) 54 Fast 

(3) 118 Slow 

( 4) Ve ry slow 

Comparing -your writing skills and your speaking skills, would you say 
you were more persuasive when 

(1) 11 Writing (2) 3g Speaking equal 
Hew would you describe your typing^ ski lis? 

m 3 None 

(2) iq Hunt and peck 

(2) Casual (rough draft with errors) 

UU) ?? Good (can do 25 v. p. to. error free) 

fS) 1? Excellent (can do 40 w.p.m. error free) 




I think computers are 



3 : y * 



Wonderful ^Stfeutral) Terrible 

31 33 14 8 2 

Have you usad computers in a batch mode for (check all applicable) 

(1) 7 Have not used them 

(2) 19 Information retrieval 

(3) Writing program* 

(4) ^74 Running existing programs 

(5) 12 Othar (specify) 



Have ybV ipFcTf 1ST programs xo 1» written by someone other than yourself? 

« * 

(l) go Yes (2)_23 — No 



A12 

37 D 



Tot. 
92 



8.. Uayu jOU ever utilized a computerized message system, tele-conferencing 
or computerized conferencing system? 

^ (1) 27 Yes (2) 65 No 

CTf yes, please indicate below which sys.tems you have used). 



87 



None = 63 
Planet-Forum 



Arpanet - 2 
Confer = 1 



Other * 13 
2+Others = 5 



9 

8 = 1 



How often have you used computer terminals for: (Check one) 



91 
Score* 

7 = 5 

8 = 8 
9=6 

10 
11 
12 

13 
14 
15 

16 
17 
18 
19 
20 
21 

t 



Never 

9. Text editing 

43 


Occasionally 
(2) 

24 


Frequently 

(3) 

?,4 - 


10. Information retrieval ^ , 

"39 


3(1 


1.9 


11- Programming 

25 - 


31 


35 


12. Packaged analysis programs 

: . 2i 




9.R \ 


13. Daca entry 

1.1 


*A 


' 78. 


14. Game playing 

/ , 4(1 


39 


1? 


15. C^her (specify) 

7R 


« 


11 



8 
11 



6 

10l6. 
3 

8 
8 

2 17. 
3 
1 
1 



Have you ever utilized, on a regular basis, a TWIX or like communication 
system? Tot. 



(D_LJ_ Yes 



(2) 74 No 



37 



Please describe your access to a computer terminal at your office or 
place of work. 



(1) -4f 

(2) 25 



(3) 47 



No terminal 

Have my -own terminal 

Share a. terminal 



90 



7 

I ■ 

1 

t 

M 
■ * 

I 

: 3 

M 

'st 

I 



If shared: Own = 23 7 

I7a« On the average,' how long does it take you to get to the terminal? 

2 min = 20 6-9 = 1 No term = 15 85 

2-5 = 24 10-19 = 1 : Minutes 2 0+ = 1 

17b. On the average, how long must you wait for someone else to get off the. 
terminal before you can use it? * 



-Routes 



80 



Own = 23 
2 min = 11 
2-5 » 13 



6-9 = 1 
10- 19 = 9 



20* = 9 

v»o term = \i 



Tot 
91 



89 



80 



* - 

18. Do you have « terminal which you keep *c home? 

(2) 51 MO ' 

18*. If ao: Is there / terminal available to you that you can take home? 

(1) 25' Tee 

(2) ^_ Ho 

i 

'19. What types °of terminals do Jrou have access to? (Check all that apply) 
♦ * 1) 30 Hard Copy No hard copy = 9 

a) Speed: No speed = 9 
5 io 7 15 50 30 characters/second or more 

b) Weight: ' No wgt. = 8 

10 Under 20 lbs. 21 between 20 & 40 lbs. 

25 over 40 lbs. 

2) _£ Visual Display (CUT) No s n 

42 both , . _ . 

20. I would not trust computer storage of paperwork that I use dally. 

r «i Strongly agree 

21 1 , 





35 Disagree 



19 Strongly disagree • * 
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f Current Expectations 

About the EIES 

to* i. (a) Concerning the user iafowfrtion brochure yfrbout the EIES f checj^ < 4 
one of the following / 

(1) .3 Did not receivers brochure 

(2) ] ft Received a brochure, . but haven't read it 
89 { \ (3) M Found the brochure easy to understand ' - 

X. A (4) g ^ Found the brochure hard to understand 
. (5) IQ Read the brochure, but can't evaluate it 1 ^ 

(b) Is there' any part of the Information Brochure or one-page User s 
Guide wWlch you had difficulty understanding? (Please be as 
specific as possible, listing page or section number.) Is .there 
anything that you felt was left out? *y other suggestions about the 
brochure and/or User's Guide? ^ > 



tr 5~ 


ia 


23 6 


21 


6 12 


21 


» 7 9 


16 


8 9 


23 


5 7 


19 


3 1 


11 


1 1. 


1 



2. Which features of the Conferencing System do you anticipate as being most, 
useful to you? (Pleaae rank fultiplfe selections 1,2,3 ett^O " 

Not r'd Ranked - R* nked ^ - 4 °T better 

91 12 (1) 37 Private aessages between individuals 

13 (2) 28 Group discussion! and conferencing 

50 (3) 2 Text editing features . j 
56 (4) 3 Personal notebooks 

51 (5) Q Bulletin '^^0^ * 
59 (6) . 1 , Searching the conferencTre cords 
78 (7) Q Use of anonymous coaaenti or pen names 
87 (8) ; 1 Other (specify) 

3. How mych time in the average week do you foresee yourself using the *E1ES? 

gg ; I C^Jtil— 30 AlQuCli or lt,s 

/ (P ^n 30 ainutes to 1 hour- 
A . m . 3* 1-3 hours 

(4) 17 3-6 hours 

(5) 7 6-9 ^purs 

(6) i 9 hours pr aore 

< . . * r 
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Tot , * 

4. Bow often do you f create yourself signing on che system tp send or 

receive messages or discussion comments? 

89 (1) 2 On^e a month or less 

* (2) 9 3 times a month 

(3) 17 ; Once a week 

{4) ^43 Two or three times 'a week 

(3) ^7 Daily 

(6) . 1 Several timM a day 

5. Do you anticipate entering the material into the 'System yourself or having 
someone else do it for you? ■ • - 



92 



92 (1) |R4 Type it myself 

(2) 4 Have it typed 

(3) Both will occur 

' ' I 

6. Which statement best descri^ef your incentive for using the System? 

91 (1) * I am required to use it 

(2) II I have been requested to use it 

(3) * 7 * " I am free to use it as I wish 



7. Which of the following best describes your anticipation of the system's 
worth?, (please check only one) 

(1) _2 i think it will be useless . Jgt . 

(2) 1 I think it is usefi^l for others, but not for me 

(3) 8 I ai? skeptical about it but willing to try it „ 

(4) 2 i am basically indifferent or neutral * 

(3) 28 I think it will have limited, but some worth for me 

(6) 40 I think it will be useful in tnany respects 

I think it will revolutionize my work/communication 

(7) 6 processes 

(8) 5 It depends (specify) ^ 

8. Which Of the following do you feel will limit your probable use of the 
. system? (If more, than one applies, rank them 1,2,3, etc.) 

1 Nof r'd Rank 1 

1W ST • (1) 3 Inconvenient terminal location ■ Rank 2 



ST (2) 4^ preference for: fa$e«-to-face communication 7 

71* (3) S Preference for telephone communication 5 

65 The people I wish to communicate with are dot 4 

82 (4) in on the system ' • ■ g 

49 (5) 7 Typing skill or lack of a typist 4 

71 (I) %} Not enough time 3 
89 (7) a , System too cumbersome or difficult 7 
77 (8) 1 . Cetoeral dislike ■Prfcbfputers 0 

72 0) 1 Prefer draf ting. ?y longhand or dictation 3 
(IE) 7 t Other (specify). * 4 



? . 


4+, r'd 


4 


3 - 


5 


8' 


2 


8 . 


3 


4 


% 


2 


4 


3 


3 


S 


0 


1 • 


5 


4 


1 


6 
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9. Cr-apar'-d to the conventional Beans, of communicating with] your group, 
do you expect the EIES to • " - * 



CD 
(2) 



Invplve less 6t y^ur time 
Involve more of your time- 



(3) 11 Involve th* same amount of time 

* 

10. How do you think use of EIES will change your commuhicatAons or 

work patterns? (Mease be specific. What current activities would 
it replace?) * 1 -\ . ' - ' ' 



As 1st • 1 '•«"»•■» . - 

Answer 2 Re P lace rsrch % 4 

3 ^or hail, pjwie =16- ; 

4 ^or spec, activity = 8 . ^ 

5 ^ com men; Contact - 29 ' 

6 Improve rsrch = 2 , ■» N -» 
. , '7 r other ~= 5 . 

11. tihy l/yo.u C per§onally irLs§ to use EIES? (What.4o you Miink you, 
^or yodr group, or the society,- can gain frota it?)^ - \ »' 



s 1 



12. What disadvantage or] negative cofise^uel^ces might possibly flow 
from yotir group's use of. the system?" j x l \ 



13. "A^y other comments' 



14. How-^ong did it take yoa to f itl< in* this questionnaire? 



THANK YOU VERY, MUCH 



1 



Continuation Page 



Continuation ©Question # 



Continuation of Question # 



Continuation of Question # 



Continuation of Question # 



Continuation of Question* # 




APPENDIX B 
FIRST FOLL0W 7 UP 
QUESTIONNAIRE FOR USERS OF EIES ^ 

1 INTRODUCTION 



J 



The questions below relate 'to your current reactions to the Electronic 
Information Exchange Systenj, and to possible effects which it may have had upon 
your work and' the development of scientific knowledge within your specialty 
area. It is the second of three questionnaires which you will be asked to 
complete for purposes of the overall evaluation of the impact of EIES. 

As in all other phases of the evaluation of the EIES system, we will 
guard the oanf identiality of your replies-' A copy of your answers will be 
provided to the evaluator for your group. The data will not be released 
in an individually identifiable form to anyone else.. 

There is a continuation page at the end of the questionnaire, for any 
answers which do not fi? in the alloted space. The numbers and spaces in the 
margins are for use in coding your, answers. Because of the "protection or human 
subjects" regulations, I need to have your "written permission" to take part 
in this project. Please be sure to sign below and return the questionnaire. 

In pretests, completion time averaged only twenty minutes.^ However, if 
for some reason you do not wish to compile this questionnaire please check 
the .appropriate apace below and return this questionnaire. t 

i * 



Sta*r Roxanne, Hiltz, Ph.D. 
Associate Director 

Computerized Conferencing h 
Cpmmunicationss Center 

New Jersey Institute of Technology ^ 



I do not wish to complete this questiotfnaire because: 



J_ I agree to -participate in this study 



SIGNATURE 

« , - - 



. - .386 



I. ACCESS & USE PATTERN 

X. What are the main activities you have been engaging. in on .the EIES 
system, and with whom? <~ , • 



COL-CODE 
1-4 



7 



I J 

< \ 



r: 



2. Does anyone else use EIES under your ID? If so, pLease giVe^their- 
name and approximate on-line time per week. /' j 



Tot. . ' 



Yes = 16 



No = 87 



Other time on: 



3. In an average week, how many times do you personally "log in," and 
use EIES? Approximately how long do you usually spend per session? 



Actual 



Preferred 



Average #■ sessions per week 
Minutes per average session 



4. How much time do you spend "off-line" in *n average week doing EIES- 
related work (preparing entries, filing matferial received, etc). 



5. ' Of TrEi/tifme spent on ElES, what proportions do you spend at your 
.office, aCthome, or at other locations?* 

* ; 1 at office 

, X at home 

«% 

' • \ 

X Other (describe) 

> iooz 



5 
* 

■ 6-8 
9-1Q 
11-12 
13-14 
15-16 



17-19 

I 

20-21 
22-23 
24-25 
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cot/op! 

Tot. 
26 i£L 

27 



6. 



£Z?t£T£' pr.vto»,',u..tioo.»lr. oad your .c=... to t.r.la.1, 
< jL , ootii.o,.3 .ioc. th-r*.«k .»«» «o .*1P to ,..;tioo,7 oo th. , 
next page) • , 

45 No cbanfce in terminal access since last questionnaire. ^ 
(43%) Change = *60 (57%) 

a) Please describe your access to a computer terminal at your office 
or place of work. » 



28 103 



29 101 



30 JLCL3- 



31 105 _ 



3? Mi. 



33 



3^ 



105 
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1) 5lLlvU - No ter J dnaL 27 ( 2o« 5) No cnan S e • • 

2) 4 Q (33 ^ Have my own terminal 

3) 3 -1 {w 1) Share a terminal 

Jf shared: > 
* On the average, how long does it take you'to get to ' 

the terminal? ' (g2%) • i0t19 min = 4 (3.9) 

9 * - 12 ( 11 7%) 20+- = 3 (2.9) ■ No term 
< 2 = 17 (16. 5) Minutes 2-5-12(11.7%; (w) 

On the average, how long must you wait for someone ^ 

• Sse to get off the terminal before you ca* use it. 

Has own = 70 (69.3) 10-19 = 5_ (5) 

V2 = 15 (14.9) Minutes 2-5 = 8(7.9) ' H° term 

c) is there a terminal available to you that you can take home? f 

I) g (8) Yes no change = 29 (28) 

• 2) 27 (26) No At home. = 39 (38> 

d) What types of terminals do you have access to? (Check^all that 

no change = 29 C27.fi) 
1) ; 69 (65 .7)__ Hard Copy no = 7 (6.7) 

•a) Speed: None = 2 (2) No hard copy = 35 (35) 

. .4 (4) 



10 4U1_15' Sj m 30 characters/second 

>W _ t ^1/ • Qr more 



b) Weight:' , None = 1 
M Under 20 lbs. 



2) 



33 



42 



No term = 35 
_ between 20 6. 40 lb*, 
over 40 lbs • 
Visu3lDisplay (CRT) ♦ „ 



-12- 
24 
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7 • Currently, do you yourself type materia^into EIES,Lxi0es someone 

type it. in for you, or iio both occur? 35 HQ 

1) (ft4) Type in myself, (Answer A below) 

2) (A) Haye typed in (Answer B below) 

3) 13 (12) Both occur (Answer A & B) , > 

.A. What type of material do you type yourself? (If more than, one, 36 

rank-order by frequency). 

i 

?*kin = 76 oth. = 16 nnt r *r| = ? I type in previously unwritten thought s/ideas e.g., gg 
* I compose on line. H/p^ = 4 

Mai n= 16 oth. ="30 n ot r'd = 6 * £7P e in rough drafts from outlines or notes. 54 

N/A = 2 

Main = 8 oth. = 6 not r r d =14 I type in material that was previously written 33 

out and edited. , w/a - 1 « . , n 

y N/A = 3 2 or more r'd = 2 

Main = 2 oth. = 1 2 or more = 1 Other (describe) N/A l ^ 



Bv What , are the main reasons why you have chosen (so h^e ^scJmeone 

else input material for you? .(If more than one, please rank-order) 

Oth. = 2 ' "not r'd = 15 I don't know how to uae the system. N/A = 91 37 Klfl 

Main = 7 oth. = 1 not r'd % = 9 I don't have time to use the system myself. 38 

2 3 12. I do not know how to type. 39 



2. f 1 * 'VlJ ^* nc * using the system directly, ie., typing 40 

a terniinal, incompatible with my professional 
*ole or job description. 



I dislike working on litje (describe why in the 41 
space below) 

« # 3 Other (please describe) 42 



i 



l 



3b.j " 
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8. What, do you do witfi the print-puts of material from EIES? 



43 \ f)Q D 4 Throw them a^.1 ou4. 

. * . * I 

1) ?n Keep them all. -* , . . 

3) 25 Save selective eAtries in a single file or pile 

• *) 37 Save selective entries in separate files (please 

* i explain filing system below: by subject, author, 

^ ' group, or what). 

i 

5) 5 I use a CRT and do not generate print-but 

6) 16 ; , Other (Please describe) 



44-45 213 



0 = 3 -5 = 132 5-10 = 40 * 11-15 = .11. 16+ = 

N (1.4%) (62%) . (18.8) . (5.2%) (12.7) 

7. How many different people £o you feel you are actually exchanging 
information or communicating with on this system, currently? 




0=18 (8.5) ^5 = 143 (67.1) • 5-10 = 45 (21.1) 11-15 = 1 (.5) 16+ = 6'tt.6> 
46-47 213 8. Of these, how fany have .you "met" (g<jjtfcen to know) over EIES? 

*8 104 9. Haye you sent' transcripts or 'other material to persons outside the - 
EIES system,, invited other, persons to be informal "observers" or 
otherwise expanded participation beyond your user group? (please 
explain) • 

i 

' Yes = 54 (52%) § # ' 

NoV 50 (48%) 



J 

49 *o. Kt the present time, which of the following best describes your EIES 

group? 

41 Mote of a collection of individuals than a research cocmunity 
54 A # set of cliques or subgroups with interests and /activities 

in common, but not an integrated community 
_j ± ^11 integrated researth community that shares many interests 

and activities in common 



Y 
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II • OVERALL REACTIONS TO* THE EIES MO&E OF COMMUNICATION 



These questions relate to #our overall reactions to e the system # at 
this goinc, as a means of communication and work coordination for your 
user groups They consist of a number of rating scales on which you are 
to circle *ne number which corresponds to where you would place your own 
impressions of the system on that dimension. For -example, here is the 
first scale: 



1. Overall, the EIES communication system, is 



12 
1 



38 

2 



'30 
3 



7 
4 



10. 

5 



\ 



4 

6 



0 

7 



53 



Extreme-' 
ly Good* 



Neutral 



Extreme- 
ly Bad 



(nr (38) " " " 

If you think that the system is extremely good, you should circle 1. 
If you think the system is quite good, you should circle "2"; 3 would 
mean that -the good aspects 'slightly outweigh the bad aspeefs^ "4", the 
middle .point, should be checked only when the words at the two ends of 
the soale describe the system equally^ well. Continuing on, "5" would 
mean that you feel that the bad aspects slightly outweigh the good as- 
pects, etc. , 
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1 find using EIES to' be . 
22 41 30 

✓"2. : 1:2 :' 3 * 


t 

8 • 


9 

3 


1 

• O 


0" 

7 


3. 


Stimula- 
ting 

(19.8) 

1 : 


(36.9) 


(27) 
30 


Neutral 

(7.2) 
= 'I , 


(8.1) 
16. 


.(.9) 

; 4 


Boring 

: f : 


4. 


Productive * 

(6.4) (21.8) 
17 40 

: 1 : 2 : 


(35.5) 
25 
3 


(15.5) 
15 ■ 
: 4 : 


(14.5) 
12 

5 


(3.6) 
2 

: 6 


Unproduc- 
tive 
(2.7) . 
,0 

r 7 : 


5. 


Great 

: . V : 


(36) 


(22.5) 


(13.5K 
: V.: 


» 

(10.8) 

H 


(1.8) 
: I 


Unpleasant 
Work 

. 4/ . 


X 


Time-. 

» 8 ' 
9 

: 1 : 


(14.5) 

12 

2 : 


(19.1) 

25 
3 


(24.5) 
24 

: 4 : 


(21.8), 

26 - 

' 5 


(6.4) 

10 

: 6 


Time- 
5 

: 7 


a 


Not 

Frustra- 
ting 
- (8.1) 


(10.8) 

« 


(22.5) 


• 

(21.6) 


(23.4) 


(9) 


Frustrating 
(4.5) • 

y 












~/ 




i 

* 



54 



*5! 




58 



( 



A24 



3^ 



COL/CODE 



: ■ 7 r 

59 7 - ; * : — L - : 2 ; 2 " S " rr ~T" Impersonal 1.2644 .9303 



62 



63 



64 ± 



18 


35 


31 


7. I : 


2 : 


3 


Friendly 






18 


31 . 


24 


8. : 1 : 


2 


3 


Easy 



16 


6 


4 - 


4 


: 5 : 


6 


20 


14"^ 


3 


4 


: 5 


: 6 



MEANS? 
Std. Dev. Sig 



. . 7 . , - , . . - - 7 - ; 1.3717 .3227- 

60 8. j L_--_i • 1 • = Difficult 

Does using EIES become so demanding of your time and energy 
• , tZt it intrudes upon your capacity to engage in other professional . 
I or personal activities? « 

* 1* . « . n :_JT : 3 6 1: i_: 1.4813 .1110 

61 * 9 ' 1 uT" ' ' I'* 

. , demand- 

demand ' ing or 

* ng °\ . ' Intrusive 

Intrusive 



10. When you send a Message over EIES -rather than siting or telephoning, 
would you say 'that recipients are generally 



1) 39 More responsive^ an EtE5~"mes3age. 




2) 26 Less responsive 

3) 33 Ho difference 



I 3 .writing but < phoning m 
1. What is' the attitude of your wife, children, or other persons with 
whom you live towards your use of EIES. « 



2. Which statement best describes your present reaction to EIES ? 
(Please check only one) 



1) 


0 


I 


2) 


4 


' I 


3)- 


8 


I 


4) 


* • 

. 0 


I 


5) 


44 . 


. 1 


6) 


_Jtt _ 




7) 


_, 18 - 


I 



5) AA , 1 think that it has certain wotthwhile uses for me 
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* III. REACTIONS TO SPECIFIC FEATURES OF THE EIES SYSTEM 



COt/CODE 



It' Row valuable or useful do you currently find each t of the following- 
features or capabilities of the Electronic Information Exchange • 
System f or # your pwti communications activity? (If .you have ftot 
actually used a feature, please check "capnot say" J. 



* 

r 

Private J 
Messages 


1 

Extremely 
Valuable 


i 2 
Fairly 
Useful 


3 

Slightly . 
Useful 


4 

Useless 

T 


5 • 

Cannot 
Say 


, 66 


36 


* 4 


1 


3 


Group 
Messages 


29 


41 


'i 

» * 

23' 


6 




Group 

Conferences . 


28 


42 


. 21 


5 


13 


Private 


28 










Conferences 


24 


8 


• 6 


• 43 


Public 
Conferences 




25 


26 


9 


.33 


Notebooks 


14 


20 


8T 


7 


59 - 


Hie^ 
Directory 


29 • 


34 


20 


5 


20 


Retrieval 
Capabil^r - 


24 


23 


14 


, 6 r 


*38 


Text Editing 


30 


34 • * 


14 i 


8 ' 


. 23 


Anonymity or 
Pen Haas 




* 

11 • 


n 


31 


> 


— ■ — / 

Explanations 


12 




23 


11 


0 

* 


+ SEN 9 


10 ' 


12 


10 


2 .. 


~> • A\ 
14a* — 


String Variables 

* 


rrl — r 

13 


5 


9 


'2 




News 


17 


30 


35- 1 




21 



65 



66 



67 



» J 



68 



69 



70 



71 



72 



■- ft' 



73" 



74 



75 



76 



77 



78 



Consents or suggestions about improving these features or desirable new 



ERIC 



features? 
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79-80 



39 



4 ,Di*d -st^one demonstrate EIES to you in person or did you 
lean from the written materials? 

1) 32 live teacher 

»' . 2) 77 m written material only 



Hojp long did it take ?ou to learn to use EIES reasonably well? 
*'l-5 *'l84 "hours / 6-10 = \& 11-15 = 4 16+ = 7 

Dp.you now find "How to Use, EIES" *( on a scale of l\o 5) 

ot understandably 







51 


41 


12 


3 


1 


a) 


understandable 


1 


2 


3 


4 


5 




\ 


43 , 


45 


7 


8 


3 


»b) 


easy to read 1 


1 ' 


2 


3 


4 


5 


c) 


-* . 

well organized 


1 


2 


3 


4 


5 






27 


43 


25 


10 


2 



Suggestions for improvement of, the Documentation. 



none =4 t 

Sugg =43 * , 

-Do you currently need the users guide (one sheet) or "How to Use 
BIES" to operate the "system? 

? , ^ - , , more than 

1) 31 User's guide 2) 16 "How To" 3) 37 Nothing 1 = 20 

5b* If ybu mow operate the system from memory, how long did you 3-10 = 1 
rely on^the guide to get you through the system? = ^gg ^our 11-15 = 

16^* = 6 

Have you ever asked a user consultant foj help? j 

1) 26 No * 1 tt 

' r 

2) 4' Yes ( Please describe -whether this was helpful, satisfac- 

tory, courteous, or what). # 

Yes, good = 67 ^ Not reached =. 5 ' 

Yes, reg. or mix4H = 6 • 
How would you rate the performance of 

' 51 29 18 4 2 
Your group leader? Excellent 1 2,3 4 5 Poor 
(principal investigator) 

, 29 25 24 2 

Systems monitor Excellent 12 3^5 Poor - 
(EIES, -100) ' , ' 

• 1 . ' * ? N 
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8. Do you find the language of the system understandable? 
' * 45 43 14 3 3 

a) Understandable 1 2,3 45 Confusing 

12 3 4 5 Inhuman 
42 40 16 3 4 



b) Courteous 
9. (Direct editing cotnmands) 



18 

19 _v 



Do 'you find the use of the +, 

34 28 

Easy to remember 1 2 

Easy to use 12 

39' 36 

' Conments? 

» 



-, * (special symbols) etc to be 

22 11 11 i 
3 4 5 Hard to remember 



3 

13 



4, 
7 



5 
7 



Hard to use. 



20 
21 



10. Indirect editing commands (.text, .tabs, etc) 
< G6od 1 2 3 4 5 Poor 
9 9 20 5 8 

Comments? 



22 



11. 



Which of the following do you currently use to operate the system? 





1 

Never * 


2 

Sometimes 


3 

Frequently. 


4 

Often 


Innc 'menu 


28 


32 


15 


15 


aVinr^ menu 15 


38 


13 


28 


* - — r i — 

"answer ahead" 


2* 


2fi 


19 


16 


commands , 


17 


30 


16 


25 


atrinst variables 


62 


20 


: 2 


6 



23 
24 
25 
26 
27 



12. 



(Answer only if you have used both menus and commands) • 
Do you now think it fs ■ a good idea or a poor idea to introduce -the 
new user to the system through menus, and provide equivalent commands 
for those who prefer them? 



54 Good to use menus first 
19 Should teach commands from the start. 
5 ft Other 



13. 



zcepp 
ef er»? 



In EIES, you do not have the choice of permanently refusing to acce 
a private or group message. Which of the. following would ym prete 

51 R equire acceptance of all messages, as at present 
17 R equire acceptance of • private messages only 
~26~~A llow rejection of any message, with "message refused by ### returned 
to the sender 



1-10 
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Conments? 

Time to complete (to be corrected) 

39.;' 
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13. Thiaking back over'yoar experiences • so far with the system, 
how frequently have you felt.. (check one) 



S 



28 . 

/ 

29/J. 

30 . 

31 . 
32 



( 

\ 

Distracted by the mechanics o£> 
the System' t 


1 

Always 

<* • 


2 

Almost 
Always 


3« 

Some- 
times 


4 

Almost 
Never 


5 

Never 


6 

(5.6%) 


17 

(15.7) 


53 

(49.10 


25 . 
(23.1) 


1 

(6.5) 


Constrained in the types of 
contributions you could make 


4 

(3.7) 


18 . 
(16.8) 


'47 
(43.9) 


i 

30 . 

(28) 


8 ' ' 
(7.5) 


Overloaded witfi information 


5 (4.5) 


20 (18.2). 


60 (54.5) 


18 (IS. 4) 


7(6.4) 


Ablte'to express* ypur views 


26 (24.3) 


50 (46.7) 


26 (24.3) 


5 (4.7) 


0 


Able 'to get an impression of 
personal contact with other* 
participants 

■ Liw : — 


t 

9 

,(8.3) 



. 38 
(34.9) 


50 
(45.9) 
. 


7 

(6.4) 


5 

(4.6J 





14. — Bow satisfactory do 
activities? 


you think 


the sys 


tem is 


for the 'following 


r- 




i 

m \ 
* \ 




COMPLETELY ' 
SATISFACTORY 

i 








COMPLETELY ' 
UNSATISFACTORY 






Giving or-' receiving 


: 29 


■ 47 


= 20 . 




3 








33 


information 


1 • 
(26.9) 


(43.>5) 


3 

(18.5) 


(8^3) 


5 

(2.8) 


6 


7 


i 


34 


Problem solving 


: fi 


: — 18 


: 27 


: _27 : 


— 11 : 


— 34 1 


-•— • 








1 


2 


3 


4 




- 6 






35 


Bargaining 


' 4 


: 16 . . 


: 16- 


.-25 : 


— U : 


— 12 : 










1 f 


2 


3 


4 


5 


6 






36 


Generating ideas 


24 


■ 29 


: 27 


: 10 : 


6 : 


4 : 


3. : • 








1 . 


2 


3 


4 


5 


6 


7 




37 


Persuasion 


: 1 ' 


• 13 


: 20 


: 29 : 


' 20 : 


11 : 


2 : 


- i 




Resolving disagree- 


1 


2 


3 


4 


5 


6 


7 


i 

» 


38 


: 1 ' : 


15 


: 24' 


: 21 : 


15 : 


' 15 : 


3 # ' : 






ments - 


1 


2 


3 


• 4 


5 


* 6 


7 


• 


39 


Getting to know 


: 1 : 


23 


: 31 


: 20 


17 s i 


8 ' : 


2 






someone 

Giving or receiving 


1 

: 14 : 


2 
20 


3 

: 18 


4 

: 22 : 


5 

10 


6 

3 .: 


7 

-* « S 


i* 

i 


40 


orders 


1 


2 • 


3 • 


' 4 


5 


6 


7 ' 


i 
{ 


41 


Exchanging opinions 


: 28 : 


47 


'-ir 


: 4, : 


0 : 


2 : 


• 


i 

; 






1 


2 


3 


4 


.5 


6 


7 


! 

1 






(26.4) 


(44.3) 


(23.6) 


(3.8) 




(1.4) ' 




i 



0 
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* dt pi Arr a rHEfK MARK OR X IN ^HE APPr6p*1ATE BOX TO TND I GATE WHETHER 

' ' f AHf^ S^Ta^sVs «« Vte iH^MT SOH^UT IMPORTANT t O, COt/CO 
.NOT IMPORTANT AT ALL IN LIMITING .YOUR USE OF THE EIES SYSTEM. . 

V* 1 '* ^ 1 — * < 



REASON 



? SIost Imp: 



18 (9.3%XlNCONVENIENT ACCESS' TO. A TERMItlA*L 



3* (1<4%) ' RED. NOTEBOOK DOCUMENTATION LOOKED ' » 
- LI|E TOP MUCH TO .READ' .-, 



•yERY J- SOMEWHAT ] NOT . 
IMPORTANT I IMPORTANT IMPORTANT 



37 (19%)'Jk35 '(18%) . ! 123 (63%) 



5 

I 



llV.6%) ! 52 (26.7%) j 132 ( 67.7) 

m J — i r — 



-6 (3d) THE SYSTEM IS TOO COMPLICATED., 
5. (2.6) 



I 17 (8.8%) i 4$ (25.4) •• i .127 (65.8) 



TROUBLE WITH PHONE 



6 .(3.1) - TROUBLE' TELENET' 



19 (9.7) 1-33 .U6.8) > 144 ,(73.5) 



11*(5.7) 

14 (7.3)| 

1*7 ^8) ' 
j^(2,l) 
(1). 



COST OF TELEPHONE TELENET*, 



HAD SOME BAD EXPERIENCES 

(SYSTEM CRASHED OR DID N0T SEEM 
TO WORK CORRECTLY) 



17 (9) 



3 



30 (15.3)-. ! 38 nl'?.4) , ! 128 (65.3) 



22 (11) 



156 ^80) 



21 (11)- 



61 (31) 



113 (58) , 



7 
8 
9 
10 



11 



LIMITED NIGHT OR EVENING HOURS". 



^38 (19.6) 



- I DO NOT LIKE TO TYPE 



I DO NOT LIKE USING A COMPUTER , . • 
SYSTEM LIKE THIS ~-. .' Q 



10 (5.1) 



5 (2.6) . 

* 

3,(1.6) 

; 

67 (34.7> 

9 (4.7) 

0 (0) < 
23 (11.9) 

Tot. 193* 



THERE IS NO ONE ON THIS SYSTEM WITH 
vWHOM I WISH TO COMMUNICATE A 
GREAT DEAL . 



6 (3) 



40-(20.6) 



116 (59.8) 



-30 ( 15.4) ' 1 155 ( 79.5 



15 (?) 



173 (39) 



I AM NOT VERY INTERESTED- 1 IN THE '<•*• 

SUBJECTS BEING DISCUSSED 9^ 



OTHER PROFESSIONAL ACTIVITIES MUST 
TAKE HIGHER PRIORITY « » 



13 (7) 



11 (6) 



93 (47.4) 



THE CONFERENCE COMMENTS OR MESSAGES 
I HAVE RECEIVED DO "NOT SEEM WORTH 
READING 



INADEQUATE LEADERSHIP OF THE GROffP 



iJf (7) 



io (d 



31 (16) 



33 (17) 



151 (77) 



151 (77) 



59 (30.1) 



61 (31) « 



44. (22.4) 



9 



121 (62) 



31 (17) 



146 (78) 



OTHER (PLEASE DESCRIBE) 



39 (57.4) 



.7 (10.3)' 



18 (26.5) 



NOW, PLEASE GO BACK AND CIRCLE THE SINCI. 
COMMENTS OR EXPLANATIONS? 



ssr.'h Mnsr IMPORTANT, FACTOR. 



12 . 

13 . 

"1^ 
15 . 

17 . 

4 

13 
19 

20 
21 
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V* Conclu^on -\ 



1. Are there any ideas, Which you ire using or working 1 with at present, which 
you first learned of. on filES? (Please- try to be specific about what you 
read-and what impact it has had on your v^rk) . 



/ 



2. Are you working on any projects or papers at the present time which have 

been* advanced by your use of EIES? (Again, plea^ try to give us some speci- 



fic details.) 



• \ 



3. Are you coauthoringter collaborating closely with any members of EIES at the 

present time, using the EIES system? If so, please describe who you are colla- 
borating with, on what, and how you are using EJfS in this effort. 
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Are there an/ "new uses" you have invented for EIES, that are helping 
jrou in jour work? These uses might not be related to the , specif ic 
purpose of your group, but we would' like t& know about them. For example, 
you might use it to communicate with your family while away on business 
trips* To coordinate face to ,f ace meetings or conferences with other 
EIES members. *. * * 



5. Overall, what would you say have been the maj^n negative ttSpects of use 

of EIES for your group this far? What things' that you wish to accomplish, 
hay$ not occured, oj^what undesirable things have occured, that might be 
attributed to characteristics of communication over the system? Please 
explain as ftelly as possible.- 1 



1 



* 



6. How long did it take ]rou*to complete this questionnaire? 
Any additional comments? , 



> 



9 



3&J 

( r 
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Continuation Page 



Tro« 

question # 



From 

question # 



9 
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APPENDIX C 

POST-0 SE QUESTIONNAIRE * . / j COL 



" * ' EIES ID 1-4 

I • i 

Part I: Tour EXES Group's Research Specialty 
h four specialty group la 



5-6 



Number 



1. Is there a commonly accepted "intellectual mainstream? io the 7 

specialty? * # ^ ,> , 

v . " • * ' 

(1) 51 lea (2) • 50 , No \ 

2. To what fjrtent do you feel' that you, and thoae with whoa you 8 

collaborate are in the recognized intellectual ^mainstream" % 

of the specialty, or conversely fceel you are ^isolated" or ■ * 
"peripheral"? • * 

B (ciyle one) ^ 

Neither in v ' 

^Completely "i* Somewhat in ^ the' Mainstream Somewhat -Completely (no mainstream, 
the Mainstream the Mainstream nor Isolated Isolated Isolated < 
1 2 3 4 5 

16 2? ■ 34 X 15 0 . 10 u 

Bow would you rate the 'degree or Intensity of competition within the 



research specialty? 

Nonexistent 



w 


Intense . 


Moderate 


Low 


Intense 








1 


2 


3 


4 


2 


'15 


52' 


29 



4 v * 

3. What are the reasons for this flbnpetitiom? (Check all that apply). 

/ Scarcity of or conpetition for funds * . - 10 

Ural groups of collaborators 11 

High achievement or success drive 

of persons in the field . 12 

Some persons act unethically 13 

- ; Strongly opposing views 1* 

:. Other (please describe) : u 



ERJC 



401 



COL 



16 

17, 

I8t 

19 

20 

21 

22 

23 



4. Pleas* list the four major or outstanding people in the entire 

research specialty (not just those on EIES), and the extent to which 
you currently know them. personally and/or are in direct contact kith 
them? 



(on EIE?)y 

y*49 f n= 
y»41 n= 
y»37 rr 



b. 
•29 

■3* 



Extent of Current Contact 









Occasion- 






* « 


Cons tant 


Frequently 


ally 


Rarely 


Never 


5T»2.85 


13 
1 


. .I 2 ' 


24 
3 


21 ' 
4 


7 
5 


7-3.03 


13 
1 


i 6 


25 

3 - 


21 
4 


12 

. 5 - 


X»3 . 22 


7 
1 


i 5 


2 \ 


V • 


V 


t * 
*»3.42 


5 
1 


V 




V 


y 



24-25_ 

26-27_ 
28 



29 



30 
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^. Considering all current personal communication nodes, what is the 
total number of different individuals within the Research specialty 
with whom you are currently in .contact? % ; \ > 

6. How' many of these are on EIES? x*I0 . 78, Media n*6 . 4 3 

S.D. » J.2.7 1 

7. At the present time, which of the following best describes your EIES 
group? ^ > 

42 (1) More of a collection of individuals than a resear^k community 

43 (2) A set of cliques or subgroups .with interests and activities 

in common, but not an integrated community 
1 A (3) a well integrated research community that shares many interests 
and activities in 0 commen 

8'. Has EIES helped to clarify any theoretical controversies in the 
specialty area? 

q (1) yes, a great deal 
23 (2) yes, somewhat 

so (3) no 

If yes - please explain briefly the theoretical issue which you 
think has* been clarified through EIES discussions, and the 
extent to which it has been resolved. 

» 

9. Has EIES helped to clarify anj methodological controversies in 
the specialty area? 



/ 



4 (1) yes, a great deal 
35 (2) yes, somewhat / 

*° ^ * if y €S - please explain the methodological issue which has most ^ 
benefitted from EIES discussion, and the extent to which you think 
* the i4SuW ha* beeq resolved. / 
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Part II Tour Work, \^ 

1* Please list the names of any persons with whom you have co-authored or 
collaborated in research (colleagues- both on and off EXES) since the time 
you began using EIES. ' f 31 

K 0 9 23. 3 » 11 

1 » 20 v 4-9 » 17 

2 » 10+ - 4 

2. Professional Publications (please try to give exact numbers published in / 

the last year or underva^ : ^' (means) 



Currently in 
Progress 



Text books . 24 



Other books .45 



Journal articles , 3«1 



Papers presented 



2.4 me ^l n 



Other : .66 



Published in 
Last Tear 

.03 



22 



3.1 



2.7 (median », 1.5) 



1.7 



3* How well knpvn is your worifc*^rithin your specialty area? 

8 15 10 23 24 14 10 ' 

x = 4. jL l : 2 ; 3 ; 4 ; *3 : 6 : 7 . , ;■ 52 

Practically -c. Average |anked 

unknown / *t top 

of Field 

For the statements below please circle the response which Indicates 

your 'degree of agreeaent* / 

4. Use of EIES has Increased ay productivity-^ terms of the Quality of 
work recently completed pv underway* J 

Strongly Neither Agree Strongly 

i Agree Agree nor Disagree Disagree Disagree m 53 

1 2 ' 3 4 5 

x * 3.05 5 33 30 21 ■ • 14 

* \ 

5. Us* of EIES has Increased ay productivity in terms of tha quantity of 
work recently completed or underway. 

Strongly neither Agree' Strongly 

Agree * Agree nor Disagree Disagree Disagree 54 

.1 2 ,3 4 5 

If - 3.23 * 5 23 33 27 15 

6. U»e of EIES has increased- ay '"stock of ideaa" that might he used in 
future work. 

Strongly. > Neither Agree' . . Strongly 

Agree Agree nor Disagree Disagree Disagree* 55 

1 2 3 4' 5 • 

O X 19 . 53 9 13 9 
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« 

■ 7. EIES has changed my view of how try own work relates co that of 
Ji others in my specialty* . 

Strongly Neither Agree Strongly' 

56 ' Agree Agree nor Disagree Disagree Disagree 

" 1 2.3 4 5 

11 37 2t 22 f 6 3T-2.76 

8. Participation In EflES contributes to* 



57 



59 



60 



63 



a) Short tern professional advancement In terms of my current 
employment a 

Strongly ^Neither Agree— Strongly 

Agree Agree / nor Disagree - Disagree Disagree 

1 2 3 4 5 



6 --25 ' 30 26 15 1T-3.19 

* b) Short term professional advancement* In terms of my ^status 

among my peers In my specialty 



Strongly -* Neither Agree Strongly 

58 Aaree Agree nor Disagree Disagree Disagree ft 

"I 2 3" ' 4 . 5 ' 

I 7- 35 37 13 9 *»2.82 

c Long term professional advancement with respect to employment 



Strongly Neither Agree Strongly \ , , 

Agree , Agree nor Disagree Disagree Disagree 

i / 2 3 4 5 

-3 29 37 .1^- 13 *-3.09 

d) Long term professional advancement with respect to my status 
among my peers In my specialty 

Strongly Neither Agrem^- ' Strongly 

Agree Agree nor Disagree' Disagree Disagree 

1 2 3 4 5 

35 1 39 12 v * £=2.79 

9'. EIES has provided me leads, references, or other .Information useful 
^ In my work. , a, . 

Strongly ITslthmr Agree . * .' Strongly 

61 xgr., . Agree , nor Disagree disagree- Disagree . 

30 51 8 11 S c 2 7-2:79 

10. EIES has Increased the familarity of others with my work. 

Strongly Neither Agree Strongly 

62 > Agree Agree nor Disagree Disagree Disagree 

1 2 3.4 5, 

*«. < .Ju 34 11 3 x*2.54 

11. EIES 9 has changed^ understanding of the interest, and^/or .activities 
of others in my specialty. " ' / 

Strongly Neither Agree l^^l 

Agree Agree nor Disagree .Disagrse Disagree , 

1 2 3 " 4 5 . 

14 47 27 , • U . K.. S-2.W 
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12. How many different people do you feel you are actually exchanging 
information or communicating with on this system, currently? x» 10 . 7 64-65 

13. Of these, how many have you "met" (gotten to know) oVer E1ES? x»5.^ 66-67 

14. Compared to the conventional means of communicating with your group, 
has EXES 

(1) ' \& -Involved less of your time 

(2) 48 Involved mbre of .your time 

(3) 11 Involved the same ampunt of time 68 



COL 



15. Has the use of EIES changed' the amount of your use of other media in the last year? 



Medium / 


Increased . 


No effect 


Decreased 




telephone 4 23\ 


63 


23 


69 


* 

mails ^ * 






, .14 




f 

ttavel to professional meetings 


10 


78 


• 11 


71 


1 visits with researchers in 
othV^ location* 


U-^- 




17 


72\ 


reading .journals or books 


_2fl 


64 


— ^-a 


73 



16. Has the use of EIES affected your communication with any of the following? 
Colleagues at your institution or organization. 



25 (13 Increased 
4 (2) Decreased 
72 (3) No change 




17. Colleagues in your specialty but not on EIES 

,f| 26 (1) - Increased 
2 (2) Decreased 



73 (3) Ho change 



74 



75 
*-76 



18. During the year or more that you have been a^ambfr of EIES, have you . 

noticed that ,lt hfs had any impacts on the w&y in^ which you think and work, 

in general? 

Ho i s 77-78_ 

Tern ^ J , 

If yes — , please describe these Impacts 'in as much detail as possible. 



79-80 



COL 
1-4 19. 



Communications with researcher* la other discipline* or specialty 
aria* ^ 

45 (1) Increased 

] (2 ) Decreased 

54 (3) No change 



20. 



Comparing my contribution* or effort put into EIES y£th the amount , 
of inf ormat ion received, If Ml that I have 



r 



5 

TT 



ML 



-23- 



-20- 



1) Contributed significantly mofre than I have received 

2) Contributed more than I have received 

3) About efcual 

4) deceived more 

5) Received signif icao^y more than I have contributed 



21. 



How satisfactory do you think the system 1* for the following activities? 

(circle one) 



7 g»2.4 3 

8 g«3.9 2\ 

9 2-4X 3 

10 g»2.7 7 

11 X-4.2 3 

12 3T-4.0 8 

13 5t»3.2 5 

14 g-3.2 1 

15 g»2.3 4 

16 g-3.30 



17 S-3.5 4 
18 g«3.8 6 



COMPLETELY 
SATISFACTORY 



Giving or receiving 
Information 

-Problem solving 

Bargaining, 

Generating idea* 

Persuasion 
O 

lUsolving disagree- 
ment* 

Getting to know 
•on* cm* 

Giving or receiving 
order* 

Exchanging 9 opinion* 

Exp re** inf po*itive 
information 

Expressing negative 
^Emotion* 

Sociable 
relaxation 



7 - COMPLETELY 

UNSATISFACTORY 



24 
1 


41 

: 2 : 




10 
4 : 


7 
5 


3 

6 : 


0 

7 : 


3 
1 


15 

: 2 : 




28 
4 : 


22 


7 . 

.6 : 


4 

7 r ' 


5 
1 


8 

: 2 : 


14 

3 


25 

: • 4 : 


5 


• 8 
6 : 


8 • 
7 f 


¥ 


: 3 ? : 




, ! •= 


I 


, V = 


) •: 


\ 


: 5 : 


¥ 


: I 7 : 


¥ 


, ¥ : 


9 : 


\ 


: ? : 


¥ • 


: J 1 : 


¥ 


, V : 


1 : 


5* 
1 


29- 
: 2 : 


33 
3 


*3 
: 4 : 


7 ■ 
5 


7 

6 : 


4 

7 : 


9 

1. 


33 

: 2 : 


13 
3 


17 
: 4 : 


7 
5 


5 

6 : 


6 

7 : 


25 
1 


.41 
: 2. : 


19 
3 


5 

: - 4 : 


" 5 
5 


. 1 

: 6 -t 


2 


, 7 

- 1 


24 

: 2 : 


32 
h— 


15 
: 4 : 


8 
5 


4 

: 6 : 


6 

7 : 
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22. Pleasa estimate the maximum you vould pay for EIES under the coadltloas 
described tad how much, you vould use it • * 



Cost in 
Dollars per 
flour 


Hour* of Use 
Per Week 


# 

EIES with current" v » 
mafcbarship if 
a) Financed from ypur 
pocket 

$2.40 


V 

. b 


b) Financed by another 
source 

$6.38 \« 

■ r 




EIES 'with peer group 4 

of your choice, if 

a) Financed -from your $3.58 

pockat ' - ^ 





b) Financed by another 
sourc. 



$8.50 



19-20 

* 21-2? 

23-24 
25-26 
27-28 

, 29-30 

31-3 2^ 
33-34 



23. 



What on* or two factors. beat explain why you have not 



ed EIES more? 




35 
36 



24. How many hours do you feel It took you ." . / ^ 

a) To learn the basic mechanics of sending and' receiving messages 



and coa 



hours (median * lr84) 



b) To feel comrbrtable communicating with other*, using this 
medium hours 

To learn the advanced features which you wanted to use ^_ 



37-38 



% 39-40 

41-42 
hours "43-44 
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XII. tactions to Specific Features of the, EIES Systea 

• * 

1. How valueble ••*» «° 122. currently find each of the following 

fututM or capabilities.? 



r 



I PreauenCY of Use 

(1>~ (2) < 3 > 

Frequently Occasion- 

•iiy 



Never 
Used 



Extreaely 
Valuable 



Value 

(2) (3) ' (4) 

Fairly -Slightly Uaeleaa 



(5) 
Cannot 
Say 



Private Mesaeges 




45 
46 
47 

« 

49 

50 . 

51 
52 

51 
54 

55 
56 

57 
58' 



60 X 

61 _ 
62 



63 

64 „ 

65 J_ 

66 j 

67 

68 _ 

69 

70 , 

79-80 - 09 
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(1)' 



(2) 
Occasloa- 



(3> 
Never 



(1) 



* (2) 



(3) 



(A) 



(5) 



Useless Cannot 



5 




ally * - 


Used 4 


1 Valuable 


Useful 

-si 


Useful 


e 


Say 

1 • 


Synchronous 
discussions in 

conferences 


ft 

I 


9 9 
let 


72 


8 


11 


- - 

16 


2 


60 


System commands 
(e.ge» +cns) 


J4 


JO 


9fi 


37 
j i 


24 


7 


1 
X 


1 

24 


User defined 
commands (+&efine) 


J 


9 ^ 


/ U * 


. 90 


1 5 


a 


ft 

u 


* 


Anonymity or 
Pen Name • 


0 




0 / 


l n 
xu 


, 1 9 

X a% 


-~\ 

l 

xo 


f 

\ 2 

X £ 


45 




Uaer consultants 
and/or HELP (110) 


1 9 

XeC 


fit 


91 

aVX 


et7 




7 


3 - 


19 


Text editing (direct.) 
U-g; /old/new/;*) 




9Q 


91 
*(X 




19 


9 

*> 


i 


23 


Text editing , 
(indirect) (e.g»; text)- 


12 


20 

-* 


■ 63 


*, 

20 

•* 


17 


2 


<*- • 

0 


57 


Games (e.g. +story) 


1 i 


32 


63 




. V 


19 


U 


57 


Special programs 
'{e.g. +terms; 
^respond) , 


1 




76 


9 


9 


7 


0 

I 


72 


Interact 
programming 


1 


5 


88 




3 


X 7 


0 


80 


Terminal Control „ 
features (e.g. 
+Left, +page;4*lp) 


5 


25 


66 


I .10 • 


18 


V 


• 0 


63* 


Graphics routines 4 


3 


92 


10 


1 1 


5 


2 


1 


81 


+SEM and 777 


-K ■ 

.15 

* > 


25 


■ 55 


| 18 


19 




1 8 V 


2 


50 



1-A-ID 



5 
6 



7 
8 

9 
10 

11 

12 . 

13 
1A 

15 
16 

17 
18 

19 
20 

21 
22 

23 
2A 

25 
26 



27 
28 
29 
30 
31 
32 



Tsilored Interfacea 



8 



75 



22 



'5 
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a, ^ ^ * 

2.. Are thare any particular fiwum'of B1ES you have*found to be (Pleasa 



daaerlba and coanaat) 
a) Unique and valuable to thla typa of system? 



b) 



0„U«, distracting and/or out of . placa In thla type bf system? 



\ 



c) What ganaral Improvement* /new featuree/changes would you Ilka to 
auggaat for EIEST 



3. EIES la now at th. .tag. wh.r. cartain Individual uaera and 8^P*>" 
constructing apaclallxad Interface, and data structure- Do you^ fcow «n 
e^p«ticuler item, of thla nature that would have been particularly 
benaficial to your group? . 



4/ Any other co«nanta on the EIES ayatem or It." i«P«cta, or on thla 
questionnaire? 



4l^ 



ERIC 
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RESPONSE RATE, EIES 'QUESTIONNAIRES 



GROUP. 

PRE-USE V 
SENT * . 
RETURNED 



30 



30 
15 



35 



35 

; «23 



40 

40' 
32 



45 



35 
22 



54 



35 
8 



SHORT P-UP 
SENT 
RETURNED 



10 

■9 



14 
12 



20 
16 



31 
26 



26 

' JL2 



•LONG POLLOW-UP * 
SENT \ 2b\:' 35 37 27 15 

RETURNED 22 24 30 21 ,9 



POST-USE 
SENT 
RETURNED 



25 
19 



30 
24 



42 
31 



30 
19 



30 



0 

ERIC 
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APPENDIX E 



SAMPLE USE> CONSULTANT PILE REPORTS , EIES 1 

1.. SUMMARY OP USEE INQUIRIES POR FEBRUARY, 1978 
During the month of -February, one hundred and forty one 
interactions between user donsultants and use^cs were logged. This 
log includes the problems addressed to the. user consultants, and the 
responses to them. * The log was established by Roxanne Hiltz to serve 
as an unobtrusive way of collecting data on user problems, out of 
which data could emerge a basis for making decisions regarding the 
nature of and priority of improvements in documentation and*<other 
features needed for the EIES system. 

The main problems encountered are similar to thbse^of earlier 
months: " / * - 

1) There were fifteen problems with the use of the various 
commands for copying in and out of the scratchpad (&<M12345, +cy 
C29C40, +cya nltfi p28, etc. ) , This material is. considered an 
■advanced feature*" and is hot described, in "How to Use EIES." 
However, since various versions of the system were initiated during 
the month, even experienced users were caught unawares by the -Changes 
in specifications, such as whether or oot the § sign should be 
included in a command. ' ■ 

2) A related problem involved seven requests on how to use the 
storage areas. Their usage is, briefly described in the user 
materials; unfortunately, the examples given do not work vith current 
versions of. the system: • 

3) Eight more new users reported the "raystterious problem of 
double printing." More instructions telling how 1 to set the terminal 
for half duplex and informing the user that double printing means 

• that something is not set £or half duplex need to be included in the 
next revised version of the .basic user materials. 

f 4) There Were ten problems with, the use of notebooks; which are 
a feature not specifically documented in the existing user materials. 
Involved in these were five new users who assumed that one gets a 
personal notebook automatically. One suggestion is that either 
Murray Turoff or the System Monitor send a message, waiting for all 
members when 'they first sign on, instructing them as to how to 
request a personal notebook or conference from the Systeta Monitor. » 

5) There were several users at the beginning of tlfc month 
complaining that they did not know how to find out what conferences 
were going' on in the system. One of them sent a marvelous 
description of the "Catch ?22" .situation: 

KEYS:/I WANT TO JOIN/ /"^ 

• IN ORDER TO GET MESSAGES FROM A CONFERENCE, YOU MUST BE A 
' MEMBER. I 
A - IN ORDER TO BECOME A MEMBER, YOU MUST GET THE OK OF THE 
CONFERENCE MONITOR. \ 

IN ORDER TO FIND OUT ^HO THE MONITOR IS, YOU MUST QUERY THE 
' SYSTEM ABOUT THE CONFERENCE. 1 

BUT IN ORDER TO QUERY THE SYSTEM ABOU'f A CONFERENCE, YOU MUST BE 
A MEMBER. .' 



THUS IN ORDER TO JOIN A CONFERENCE} YOU MUST ALREADY BE A 
MEMBER* 

#*##-*##*#######gGH. 1 1 1 *************** 

* I WOULD LIKE A LIST OP ALL THE CONFERENCE TITLES AND A LIST OF 
CORRESPONDING CONFERENCE MONITORS SO I CAN ASK TO JOIN -THOSE THAT 
LOOK INTERESTING. , 

This problem was resolved by setting up Public Conference 1008 
' for a listing and description of all conferences on the system that 
others m*y asjt to join and by Tiavi'ng the group moderators send out 
messages to their groups reminding them of tehe various conferences 
and moderators. * 

6) There were eight "bug - reports, which were referred on to 
the programmers. 

No othetj problems were reported more than jtwice. 

S Resolution of problems one and two is now being discussed. 



2. Analysis -of User Consultant frile 



Items 889-1127; Aug 19 through October 31, 1978 

Roxanne Hiltz 

Dating this period , many new users became active on the system. 
These included the initial members of th^ three new DIST groups , the 
Mr. Pit group, and the two student groups , replacement or new membets 
of 'bid groups, and some individuals* In reporting and categorizing — 
inquiries to the user consultants/ a rough division has been made 
into '"new* and "old* users. 1 ANy recent new member is considered a 
•new user*, and this was determined by having an id for a new group. 
IN other cases , persons who have been on the system but have not -used 
it much are also considered "new -users*'. Any person who did not look 
familiar was~ checked in the listing of usage stats and considered new 
if total time on' line to date was less than five hours. 

( 4 

Also separated out are the . requests of new group leaders and 
facilitators, so see if their needs are notably different from those 
of other new members - * \ 

Let us take the new users questions first. *^ 

1. "Somebody talk to me. . • ■ 6 ^ 

Some of tae* variations on the request of first time users for 
attention and immediate feedback are t most interesting in their 
wording. Exs 

■Is ^anybody out there reading this now?., tell s me something. 
ANything, so 1*11 know somebody's out there and I r m -operating this 
blank thing right I* 

■This is the first message of ^ wandering wordsmith caught in a. time 
warp.* f * * * 

2. Spw can I delete conference . comments entered - mistakenly, and 
resend -them as messages? 5 

Copying item^^S 

tow "do I communicate in real time? 4 * 
How do I send a message to X (what is his name or number?) -^4 
How. to enter a conference corament-4 
Editing , Directory entry-3 ' 

How do I get a list^of conferences that can be joined?-3 
How to reset a conference marker-3 ^ 
How to retrieve ol^l messages-2 
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Baa not received* red *manual-2 
empty or unclear ?*?*s-2 

Other New User Inquiries \ 
Bow to send a message 

Bow to delete redundant rip cod* from directory 
Truble reading selected messages from waiting queue 
Trouble with .talis 
Bow tcr get back to single spaqing 



Am I on a time budget? 
Setting margins ' 
REseiting conference markers 
Accessing a notebook 
Bow to turn off menu printing 
Can you get. back 4 NP mistakenly erased? 
Using SA's ^ ^ 

, Bow to contact a pC v 
Bow to find messages 

User who had been using system without reading docummentation 
wondered why he had received no answers yet to. his messages, 
tow to enter Director^ description 

7 ' t 

BOurs of .operation 

BOw to set up a private conference 

User, from abroad sent a letter with transcript showing problem. A 
minus sign as first character was preventing entry of item. * 
User lost in system. Kept getting th^-s^me question (Modify item ♦) 
and did not knowhow to answer it. 
Bow do I correct errors?? 



. Bow transfer a short paper from SA f s to a\:onference 




v How to get into a conference *' 
Can a user consultant be reached by telephone? _ / 

How to delete a message ' , %i ' 

How to erase the SP 

' What is the meaning of "associated comment? 11 

What happens during disconnects? % 

j . 
Wha to do when the system v crashes /* # 

What does rolled mean? , \ 

Deleted too Vany items from a conference. Can they be put^back? 

Right margin wrapping around 

Why £oesnt TELENET type a disconnect? ^ ; 

Why does my terminal error light go on? 

HOw do I get to the m end of the message when I ap^Einished editing? 
Why ^oes my' phone ■ disconnect every five minutes? 
Why cannot my nidcname be*X ( It was taken) 



How to delete a gorup message 
How do I fin<j my messages? 
How "do I get a notebook? 

Can youlist conference cbmmments bagkwards, starting with the most 

recent? ^* 

Where is there" a dictionary of commands? 

WAnts not to receive messages previously selected from waiting queue 
Missing ends of lin^s ( incorrect right margin setting) 
Note that in many ca^es the problem descripton is so vague and 
confused that the user consultant mi»t first find out excatly what is 
the matter, or suggest several possible diagnoses and treatments. 

Leaders' and facilitators of New Groups 
SEtting up groups and conferences-3 

Getting a list*ng~~of class one vs. class 2 and time allocated and 



V 

A 



... 418 
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HOving users frop class 1 to class 2 * 
••What is the 800. number for TELENET? 

Confusion in adding people to conference vs. group ' 

« 

Difference between a group coordinator and A conference moderator 

» 

What does rolled mean? 
Use Link 

HOw to get into qlQ08 v 

9 * 

Trying to send a message to a conference*. 
Double letters printing 

fa 

Bow to us* Search choice 

Setting .tabs \^ 
How to put an item in both notebook and group message 
Why is repsonse to ++6,5,5, and ++6,5 f 9 similar? 
How to send a gorup message 

Overflowing lines " , 

WHy d#d a participant who received conference items get shown as none 
read? rj 

Note that the requests listed first are probably peculiar t6 the 
duties and prdblems of a group leader or facilita>tior t wrule the rest 
are much like the questions of other new users. _ 

Experienced Users 5 # * f 
Resetting conference markers-7 ^ 

Hotrf to get bid Chiaos-6 ( 

.. ^ ' ~ r. ■ • 

Request for setting form f ee<fc page contrqls-*3 ( coming soon; +lines 
command ) 

Problem with writing Interact* program -3 ( same user with 3 different 
9 • A50 



ERIC r < 



problems) 

( NOte: most questions and answers on interact are taking plac e in 
conferences devoted to specific design efforts! t 
Bow to get GAPC to work ( +s£cm)-S 

Time expiration ( "I am being banished from society. ") -2 
How to. enter executable lines into the SP 
Getting proper margin controls 
Using SA's ' V. 

??? request of th* month:* Fire here and my house is 
threatened .Please msg me when you * 



e 



people brokering 
explaining norms of use of system # 
Pen names vs* nick names 
•endtext vs »notext 1 
How to underscore 

Travelling user wanted local TELENET number » 
How to use* comands to get SA's ( you cannot) 

How to, get an Sff line printing of a three months backlog? ( system 

monitor said he'd do it) 

Deleting and adding conference members 

Changing the 'name of a conference 
* * 

Bvaluator wanted more Retailed usage stats for his group 

Retrieveing lost conference comments 
.double printings 

♦doctor seems to behave in a bizarre manner 

wants +linl to work correctly 
/ edinting items 
, experiment with batch input with paper tape 

Trouble with defined command ' 
O * , A51 
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Can members be searched by pen name? (no) 

Can the system be searched for all exi*t8ng conferneces? (no) 
Modify keys 

conference status review 

How to send a/message toa long list of addressees 
Regreiving old messages 

Request for search choice \ # 

receipt of a message in the queue was 'causing problems 
trouble with fget 

Can a copy of a very old message be retrieved from tape archives? 

How to get into somebody else r s notebook A 

< . *■ 

Did not understand prompt at modify items choice^ 

changing conf erenceNqiembership 

batch input from smart terminal 

How to see who's on line in the middle of a sessiion? 

Note that some of the things requested by useM would involve severe 

I* 

invasion of the privacy of others if such features were available 

Bugs reported to User Consultants 
♦News delivered part of a proc 

Part of a message ended up garbled into the end of lines in a comment 
being edited • '* 
gwci not working correctly* 
"++7,2 gave incorrect info -^y 
qot conference choice when" replied y to "accept new items?" 
BIBS sending out stray characters 
( non -human ) ' 
Inconsistent, gwci break behaviour 
++6,8 caused error message 
jfrong answer to ?? prompt at pen name choice 

A52 , A 
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APPENDIX F 

^ Pre-Us^r Questionnaire 

Study of the Impact of Computer Based Conununication on- 
Scientific Research. Communities 



Toi ^ 

From Starr Roxanne Hiltz 

'I am a Sociologist currently supported by the Division of Mathematical 
and Computer Science of the National Science Foundation to carry out a study 
of several groups of scientists who are using computer based communication 
systems* The principal investigator for your group, Dr, -Lawrence H. Landveber, 
has^iven permission for me to include .your Network for Theoretical Computer 
Science, using the MACC TELEMAIL system, ia the study. 
* , * ' 

The purposes of this study are to discover: 

What reactions do you as an individual have to this form of communication? 
Why will, some of you use it much more than others? 

.What effect does use of the system >*ve on your user group and your 
research specialty as a whole? 

What changes in the system itself seem advisable, based upon your 
group # # s experiences? 

The s*udy will include three questionnaires . — t>ef<$re use, at 3 to 4 
months after startup, and at the end. of the project. Each one will have been 
pre-tested and take the average person about 30 minutes to complete. I will 
also try to discuss the pifoject with some of you on a more open-ended basis, 
either in person or on the telephone, or 1 over xhe system (Roxanne=Hiltz). 
Attached is a short vita to introduce myself; 1 .will be glad to send reprints 
of any of my previous articles in this area if you are interested. 

Please be Assured that ail information collected will be treated as 
confidential. Note, for instance, that this cover sheet will be removed before 
coding. Youi/ name Qr identifying information will not be used in $ny reports. 
However, a copy of the data, with the name removed and only an ID used, will * 
be made available to Dr. Landweber for his use in the final report on your 
project. , \ 

m Completion of this questionnaire or participation in any other phase of 
the evaluation project is completely voluntary and in no way conditions your 
participation in the -TELE1AIL project itself. You may refuse to answer any 
question, and you are free to withdraw from participation at any time. I will 
be glad to answer any inquiries about the study. 

Because of the protection of human svftjects regulation under which I work, 
it is necessary for me to have your SIGNED STATEMENT OF ••INFORMED CONSENT" 
(this page) returned with your completed questionnaire. 

STATEMENT OF INFORMED CONSENT 

I have read the above and I agree to participate in this -study. 

A * 



^ . ^ Signature Date 

E£|C am 422 



Tart I. Your User Croup's fe&e.trch Specially (I.D. 



> 



What is the approximate year, in which this specialty became recognized 
(or vili^bccoma recognized) as a separate and distinct research area? 



CARD U /COL 
16 



2. 



3. 



I = 5-19 years 
7 * 10-19 years 

For approximately hof long have you been actively working within this 
specialty area? - 

4 5-9 years' 
4 * 10+ years 

What is the total .number of journals in which articles relevant to 
your specialty area are likely to appear? 

l l > 0 <_ none (5) 0 20 - 49 

. (2) q two or less (6) 0 ; 50 - 99 

3 3 - 10; , M7) 0 ,100 or more 

<*) ■ f n - 19 . 



4, 



" 9 

Is there any journal' or newsletter or other published sdurce in which 
you can find descriptions of current (unfinished) research activities 
and developments within your specialty? . \ 

(1) B No 

(2) n Yes: please list: y g 



17 



18 



19 



5. Are there any tneetingror conventions which you "must" attend in 



order to keep u> with research in your specialty? 
list). 



(IF yes, please 



(1) 



JL 



No 



(2) 



8 



Yes ( 



.20 



6. . Could you list the four major or outstanding people in your entire 
specialty aad the extent to which you know them personally and/or 
are in direct contact with them? 



a . 

C 

d. 



Extent' of Current Contact 



Constant . Frequent 
1 2 

1 .2 
1 2 
1 2 



Occasional 

" 3 



Rare' 

4 
' 4 

4 

4 



Never 
5 
5 
5 
5 



. 21 
22 
23 
24 
25 
26 
27 
26 
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8. Considering all* current personal coflOminication modes, what is-thc to^tal 
number of different individuals within ydur research specialty with whom 
you are currently in, contact? * -j_g _ 2 

* . ; « • • 11 - '2 \ 

*, J -3-15= 2 „ > 

9. How many of these are J.n your computer communication system user ^ 

group?_ _ . 



2/21 



2/22 



10. Is there a commonly accepted "intellectual mainstream" in yObr specialty? 1/33 



(1). 



Yes 



(2) 



No 



11. 



If yes; to 
collaborate an 
specialty, or, d| 

Completely in 
the Mainstream 

I*. . 
3 



extent do you feel that you and' those w^th whom you 1/34 
in the recofnized'inteliectual "mainstream" of your 
vet'sdly feel you are "isolated" 6r "peripheral"? 



Somewhat in 
the Mainstreaj^ 

—.5 



Neither in. 

the Mainstream Somewhat Completely 

nor Isolated Isolated Isolated 

3 4*5 

0 * 0 0 



A 1 * 



"How would you rate the degree or intensity of competition within your 
^research sgecialty? * 



Very 
Intense 

, 1 
.1 



Intense 



4 

S 



Moderate 

3 
4 



.Low Nonexistent 

4 5 

1 0 



^ 13. What are the reasons for this ^competition? (Check all that appty). 

v< 4 Scarcity of or competition for funds^ 
.Riv3l groups of collaborators 
>High achievement ocusuccess drive 
*f persons in the field w 
persons act unethically 
gly opposing views 
Other (please describe) : 




14. Please list the- name i>f any other research specialties in which you 
are currently invalved, and whether you are currently speeding more , 
time oc^4ess time on each one thSn on your user group's specialty. 1 



1/35 



r 



36 
37 

38 
39 

40 
41 



r 
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Name 






More 


Less 






< 


1 * 


time 


time 


42 


none 


« 1 


V 




43 


less 


time 


= 2* 




44 


r 












more- 


time 


- 2 




1 


45 / 




r 




















i 


* 
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Pure II, Your Work P*W<rfna and View of Your Specialty G^RD/COk 

1. During an avurage week, • approximately how many hours do you spend on 

each of the following, kinds of activities? (First list the total for all 



vithln your specialty area)* 
Direct research activities 


raear 
Total 

10 . n 


s 

Hours in Specialty 
' only 


Writing papers, books. >tc. 


ft i 


fi.n 


Education' 

teaching * 


10.9 


° 3.6 


-learning: reading books or journals 


4.5 N 


3.0 * 


attending' meetings, 
seminars, etc. 






Administrative at>d support 
activities (committee 

meetings, memos, etc*) 


4.9" ' 


0.25 


Telephone 

'inside vour organization 


0.5 


— 


outside vour organization 


2.4 


2/0 


Consulting 


1.5 


5.9 t 


runaxng ^grants appiicaL lctib 

or other resource acquisition 
activities) 


2.5 


/ 

1.4 , 


Other professional activities 
(please sr>ecifv) 


2.4 


♦ 0.37- 


• 

Total 


51.fi 


24 .9 . . 



( 



2. Please list the names of any persons with whom you have co-authored or 
'collaborated in research during the last year (197%78). 



-2/20 



median * 4.0 




Scientists are sometimes anticipated by others in the presentation of 
research findings- That is, after they have started work on a problem 
another scientist publishes its solution* How o'ften has this happened to 
you in your career? (Please exclude ca&es where a solution to Vour 
problem was published before you started your owft work. Circle one.) 



Constantly 
1 

0 



Frequently 
2 

0 



Time to Time 
3 

6 



Rarely 
4 



v 



Never 
5 

0 



2/23 



4. How concUfned are you that y»u might be anticipated la /our current work? 



Constantly 
1 

0 



Frequently 

• 2 , , 
2 



Tine to Time/ 
3 

- 4 



«Rarely 
4 
1 




Neyfer 



2/24 
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Central Principle! of Science 



Described below ere two sees of conflicting general principles which can 
guide the. conduct and evaluation of scientific research. Please read each 
set of principles vjith your specialty area In mind. 

Principle A* Emotional Neutrality - 

Scientists must be emotionally neutral and impartial towards their 
ideap if they are to stand a fair ^hance of) ultimately being proved valid/ 
Conducting an Investigation' with anything Lbss than an impartial frame of 
mind possesses the danger tha^jtfcan icient lit will bias results and be 
unable to give up hypo th e— fwfiey they are indeed false. 



Principle 8. Emotional Cc 



sent 



Scientists must be emotionally committed to their ideas if they are tb 
stand a fair chance of ultimately being proved valid. Unless a scientist 
believes intensely in his or her own ideas and dote Everything legitimately 
in his power to verify them, there is the danger^hat he will give* up his ideas 
too quickly. Initial inconclusive signs of negative evidence do not warrant a 
reorientation of research efforts. The scientist must believe in himself 
and hisNpwn findings with great conviction. 

5. On thev basis of your own experience and observations, to what extent does * 
ich of\the principles tend to govern the everyday working behavior of 
most scientists in your specialty? (Please circle one number). 



. A 
Signif- 
icantly 
More 
Than B 



A 

Moder- 
ately 
More 
Than B 



Both 
Tend to 
Govern 
Equally 



B 

Moder- 
ately 
More 
Than A 



B 

Signif- 
icantly 
More 
Than A 



Neither 
Tends to 
Govern , 



1 2 3 4 5 "6 

0 § 0*2 4 1 1 

6. To what extent does each of these principles tend ton govern Your own 
everyday working behavior? 

A A B^ B 
Signif- Moder- Both Moder- Signif- 
icantly ately Tend to ately lcantly t Neither 
More Mote Govern More More Tends to 



Than B 

1 

0 



Than, B Equally Than A Than A 



3 

2 



Govern 
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* Principle . C: The Irrelevancy of Personal Attributes 

The personal attributes of a scientist are completely Irrelevant in 
Judging results and claims to knowledge* Each claim in science la Judged 
i^>artiall.y on iu own merits b^ its ability to stand up to rational, empiri- 
cal test procedures without reference to the partipfclar scientist* t - 

Principle D; The Relevanty of Personal Attributes y 

» i 

The #rsonal attributes of a scientist are highly relevant In Judging 
results and claims to knowledge. In reality the work of some scien/ists 
Is given credence over that of Vfhefcs* It is necessary to know the)' personal 
chatacteris-fc^s, backgrpund and motivations of a scientist before one can 

properly evaluate his or her work* ■* 

» ' • 

As above** we wish you to indicate the extent to which these two princi- 
ples tend to govern the everyday vbrking behavioT of most scientists in vour 
specialty ; tend to govern "your owtr everyday working behavior, and ought to 



carp a /COL 



govern the behavior of scientists 



your specialty < 



C 

Signifi- 
cantly 
Mo re 
Than D 



C 

Moder- 
ately 
More 
Than D 



Both 
Equally 



D 

Moder- 
ately 
Jfore 
Than C 



D 

Signifi- 
cantly 
More 
Than C 



Neither 



Most scientists 
1 

v 0 

Your own 
behavior 




t 



1 2 

Ougiit to ^ 
gove ra 

1 \l 

I am^oore Interested \n generating* a large ?numDer of alternate 
explanations for any problem than in pursuing one exclusively 
in detail* 

Strongly neither Agree Strongly 

Agree Agree' nor Disag'ree Disagree Disagree 

#v i 2 3 4 5 

0' \ 3 . 4 , 1 0 

work in well-est^lished xes^arch areas. 

Neither Afcre^ . Strongly 

Agree Agree . nor Disagree Disagree , Disagree 

1 2 ' 3 4 5 



I prefer^<*"« 
Strongly 



How well known* is #>ur wqfck, within your specialty area? 



3 



4 v 



0 
6 



Practically 
unknown 



Average 



6 

0 

6 
0 



28 



\ 



29 



Ranked 
at top 
of Field 

0 ' 



30 



66 



70 
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Part 11V Background Items 
1. t What is your age? 



(3>IX 



under 25 
25 - 34 
35 - 44 



2. Sex: 



female 



(5) . 

(6) . 



0 



(2) 



_ 45 - 54 
_ 55 - 64 
65 & over 



male 



3.S Please list your highest academic degree (Degree, Subject,^ and year) 
t 

Ph. D. » 8 

t 

years since degree: 



2 = 
5 « 
1 « 



3 

5-9 

10 + 
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31 
32 
33 



4. Professional Publications (please try to give exact numbsrs published in 
last year or underway; estimates are fine for previous works.) 



0 Currently in 
Progress 

Text books • .9 


Published in 
Last tfear 


Published or Presented (38-54) 
during. 
Total Professional Career 

v .1 ... 


Other books *^ 


.2 


.1 


Joutnal articles ^ n 




a i 


Papers presented 1.4 




♦is :a _ 


Other (describe) .5 


1 .9 


2.S 
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IV CooiBunication Skills and 



ilities 



i. 1* English your prinary language? 

* - • ~* 

(1) a Ye* -(skip to question 2) 

,(2) ..Mo 

If not, what i* your first language? \_ 



/ n 



If English is not your first language, do you consider your English 
to be on i pat with your, primary language as to; 

»• 

(i) Yes (2) No 

' (1> Yes (2) No^ 

(1) Yes (2) No" 



Writing . 
- Speaking 
Reading 



2. How would you -describe your English reading speed' 



(2). 
(3) 

(4). 



J, Very fas$ 

Fast 

J Slow 

_Q Very slow 



Comparing your writing skills and your speaking skills, would you say 
you were more persuasive when 



' CD 5 Writing 

4. How would you describe ^yo"ur typing skills? 



(2)_3_ Speaking 



5. 



(1) 0 None 

^2) 0 Hunt and peck 

(3) 3 Casual (rough draft with errors) 

(4) 2 Good (can do 25 w.p.m. error free) 

(5) 2 Excellent (can do 40 w.p.m. error free) 



I think computers are 

: 1 ; 2 : 



Wonderful 
1 



(neutral) 
0 



Terrible 

0 



6. 



4 1 O'O 0 

Have* you used computers in a batch mode for (check all applicable) 



(1) 

(2) _i 

<3>_2 
(4) 
(5) 



Have not used them 
Information retrieval 
Writing 'programs 

5 Running existing programs 

Other (specify) 



7. Have you specified programs fco be written by someone other than yourself? 



> 

CARD 3/COL 
1-4 



7 
8 

'9 

10 



11 
12 



13 



14 
15 
16 
17 



18 



(D_S Yes 



(2). 



No 



19 
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CARD 3/COL 



8. Have, you ever utilized a computerized message system, tele-conferencing 20 
or computerized conferencing system? ' ' J 



(1) 5 Yes' (2) 2 No 

(If yes, please indicate below which ^systems you have used). 



21 



How often have you used computer terminals for; (Check one) 



Never 
(1) 

9. Text editing 

ri 


Occas ionally 

< 2 > 

2 * 


Frequent ly 

(3) 

22 

5 


^ 10. Information retrieval 

1 


6 ! 


• 23 

1 


11. Programming ^ 


5 . 


3 24 


12. Packaged analysis programs 


* 1 

3 

> 


o . 25 


13. Data entry ' ( " 5 


3 


0 26 


14. Game playing 


*5 . 


1 * 27 


15. Other (specify) ^ 


. • 0° 


• 1 28 



16. Have you ever utilized, on a regular basis, a TWIX or like communication 29-30 



system? 



(1)_Q Yes 



(2) 



8 No 



31 



17. PleaSe describe your access to a computer terminal at your office or 
place of work. 



(1) Q_ 

(2) g_ 



(3) 2_ 



-No terminal 

Have my own terminal 

Share a terminal 



32 



If shared: 

17a. On the average, how long does it take you to get to the terminal? 

4 Minutes 33 



17b. On the average, how long must you wait for someone else to get off the 
terminal before you can use it? 



Minutes 



34 
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18. Do you have * terminal ,vhich you keep at hoax*? 
ri) i ' Ye* 
(2>_3 No 

18a. If no: Is there a terminal available to you that you can take home? 
(1) o Yfes 



CARD 3 /COL 



4 

37 



(2) 



No 



Tehninals 



19. What types of t^hainals do you have access J&tf? (Check all that apply) 

1) Hard Copy 

' a) Speed: 



38 



10 



* 15 



b) Weight: 

T Under 20 lbs. 



_g 30 characters/second or pore 



between 20 & 40 lbs. \ 



_2 over 40 lbs, 



2) 7 Visual Display (CRT) . 
2ff. I would not trust coapute* storage of paperwork^that I use daily 
Q Strongly agree 

2 Agree 

3 Disagree 

. 3 Strongly disagree 



39 . 



40 



41 
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Currant Expectations 
About the Computer ttedi-ated Communication System % o 

1.' (a) Concerning the user documentation f check one of the 
# following > 42 — 

4 9 (l) 1 * Did not receive a manual 

(2) ^ 3 Received, but haven't read it 

(3) 3 Found it easy to understand 

(4) ^0 Found tt hardto understand 

(5) 1 Read it, but c§n't evaluate it 

(b) Is .there any part of t>he documentation you had difficulty under- 
standing? (Please be as specific as possible, listing page or 
section number.) Is there anything that you felt was left out? 




2. Which features of the System do you anticipate as being most # 
useful "to you? (Please rank multiple selections 1,2,3 etc.) 

^ (ranked #1) 

(1) 7 Private messages between individuals t t 

(2) 0 Group discussion and conferencing 
(3 fr» 0 ^ Text editing features # / 

(4) 0 ~" Personal notebooks 

(5) 0 Dissemination of Research Announcements 

(6) 0 Other (specify) 

3. How much time in the average week do yoi^ foresee yourself . losing the system? 

( IV. ■ 1 30 minutes or/less / ' - 

(t) 2 » 30 minutes to 1 hour 

(3) 3 1-3 hours • J 

(4) 1 3-6 hours 

(5) I I 6-9 hours 

(6) u 9 hours or more 



43 
44 

/ 45 
46 

50 
51 
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C**>3 /Cue 



4. How of^ta do you foresee yourself signing on the system to s*nd or 
receive messages or discussion comments? i 




Once a montA or l^ss 
2-3 times a month 
Once a week 

Two ot three tines a week 
Daily ' 
1 Several times a day 



1 



fx. 

5. f Do you anticipate entering the material into the^ System yourself or having 
someone else do it for you? 



(t) 7 Type it myself 

(2) n Have it typed 

(3) 1 Both will occur 

6. How strong is your motivation to participate in this system? 



Very Strong 
4 



3 

0' 



Very Weak 

0 



54 



7. Which of the following best describes your anticipation of the system's 55 
worth? (please check only one) • 



(1) . 

(2) . 

(3) . 
(A) 
<P). 

(6) . 

(7) . 

(8) . 



I think it will be useless 

I think £t is useful for others, but tiot for me 

I am skeptical about it but willing to try it 

I am basically indifferent or neutral 

I think it will have limited,, but" some worth, for me 

I thi"hk*it will' be useful in many * respects ♦ 

I think it will revolutionize my ^work/comnunication 

processes 

It depends (specify) 



8". „ Which of the following do you feel will limit your probable use of the 



systea? 



(1) . 

(2) . 

(3) . 



(*)- 

(5) . 

(6) . 

(7) . 
(«). 
(9). 

(10) 



(If more than one applies, rank them 1,2,3, etc 
(ranked #1J 



) 



Inconvenient terminal location 

Preference for f acer-to-face communication 

Preference fot telephone communication 

The peffple -I wish to communicate with arc not 

on tw system 

Typirfe^kill or lack # of a typist 
Hot enough time 

System too cumbersome or difficult 
General dislike for computers 
Prefer drafting by longhand or dictation 
Other (specify) 



56 
57 
58 
59 
60 
61 
62 
63 
64 
65 



9. • Compared tor the conventional means of communicating with your group; 
do you expect the computer system to 



ERLC 



Involve less of your time* 



(2) 3 Involve more of your time 

(3) Involve the 'same amount of time 
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Nov, please fill in the enclosed "social ties check! 1st". First read 
the category (ex: unfamiliar to ne. M ) Then, turn the papqr sideways 
and put an "X" in the box for every name to which the phrase appears.* 



10. How do you think use of the system will change your communications or 
work patterns? (Please be specific. What current activities would 
it replace?) 



11. Why do you personally wish to use the system? (What do you think you, 
or your group, or the society, can gain from it?) 



12. What disadvantage* or negative consequences might possibly flow 
fxom your group's use of the system? 



* 13. Any other comments? 

y 



14. How long did lc take you co fill in this questionnaire? 



76-78 
79-dO 



THANK YOU VERY MUCH 
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Continuation, Page 

Continuation of Question w 



I 

Continuation of Question f 



Continuation of Question # 



Continuation of Question # 



fcontlnuation of Question # 

— r 
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L 



§ 

H 



UNFAMILIAR (NOT KNOWN) TO ME 

PEOPLE' I CONSIDER FRIENDS 

CLOSE, PERSONAL FRIENDS 

AUTHORED BOOKS, ARTICLES, OR 
PAPERS THAT I HAVE READ 

FELLOW STUDENTS WITH ME 

TEACHERS OF MINE 

STUDENTS OF MINE 

WORK OR" HAVE WORKED AT 
THE SAME INSTITUTION 

WORKED ON A PROJECT TOGETHER 

CO-AUTHORED WITH 

MET FACE-TO-FACE 

CORRESPONDED THROUGH THE MAIL 

TALKED WITH ON THE TELEPHONE 

TALKED TO AT CONFERENCE OR 
AT A MEETING 

COMPUTER CONFERENCES WITH 



X 



a 1 



< 

a 

H 

I 

M 

W 

8. 

O 

w 
o 



u 




w 



§ 

u 

H 




8 
S 

H 

s 

9 

Si 

M 

W 
H 

» 
U 



I 

I 

i 



W 
H 
« 

H 
PS 



o 



CO 



I 




8 

z 

H 

>4 
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APPENDIX G 

'Study cf clie Irpact *of Computer Based Communication or^ 
Scientific Research Co<nminitie3 



^ * Start Rotfann? Hilt?:, Ph.Dy 



. To 



« T ip i .sociologist 'currently being supported by the Division* of Ifptheaatical 
tirf Computer' Science co^carry out a ^tudy of several groups of^scientists who. 
ire u* ins, computer based communication systems. The principal investigator for 
Av>ur ii roup, Or. Lawrence H, Landweber, ham given permission for me to include ' 

• m.r Network for Theorepical Computer Science, using 'the MACC TELlMAIL system, * 
i • tne i»tiidy. 

§ . *~ ' * 

TJt&.^^pbse^of this stifdy are to discover: 

What, reactions do you as an individual have ro this form of communication? 
%nv w:11^otc of you use it much more than others^ 

AhjTt effect doe\use of the system h^ye on your user group and yoylS research 
-?OL:aItv as 5 whole? ) * * 

tflgit clumgos in the system itself seem advisable, based upon your groups's 

* *>t*r i en ces .* ^/ 

Ij pre-tests % chis que'stionnaire*took about 15 minutes to complete, so ^ 
plea^f grant us this much of your time and complete and return it as soon as * ^ 
possible. If there are any auestions, you may send me a message (Roxanne-Hiltz.) 

• r .'•>'«! .me nt 201-232-6632. " . 

* "lease be assured that all informatctfen collected will be traat^d as confidential,, 
\o'i, for Instance, th.it "this cover sheet will b'e removed bef or^Coding, "Your 
inv *r identifying information will not be used in any reports. However, a copy 
e: itu Jata, with the name removed and on.ly an ID used, will be made available to 
Pr. Li. Jweber for his use in the final report on this project, 

rie.nse be assured thaq completion of this qdes tionnmirt or pirticR%)n 
i-u.mv oiher phase oXthe evaluation project is completely voluntary andnff no 
wnv conditions youfr participation in the TEL EMAIL pfOject itself. You may refuSj 

.n ^vor any questtoMahd you.artf free to withdraw from participation a*--«*-^time . 
' I *mU *k» glad to answer *kny inquiries about the study, \ 

It c\une of the protection of hurafrn subjects' regulation under which I work, 
.v Hilary for me to have YOUR SIGNED STATEMENT OP INFORMED CONSENT" RETURNED 
;»i.v).' in ortier to process ypur answer. \ 



» t i 



i iv e t u -tbove and I asree to participate in thi* study. 
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i. kctui t on FArpoH 

1. What are the main activities you havi been engaging la on the sftem, 
• and with who*? : \ # 



t 



C0L-*0DE 
1-4 



2. Does anyone else^ use the system under your ID? 
their tun and approximate on-line time per week* 

yes "= i 

i - . ' . 

3. In en average week, how many time do you personally "log ln ,v ? 
Approximately how long do you ueually spend per session? 



give 



^linutes/week^ 

12rx5 Z 
20-30 * 
50+ * 5 




Average # sessions per- wedt 
Minutes per average seeeion 



4. v Bow such time do you spend "off-line" in en average week doing 
related gg^k (preparing entries, filing material received, etc). 



5. Of the time spent on the system what proportions do you spend 
ayour offic^ at home, or at other locations?. 

♦ Z at office 



Z at home 



Z Other (describe) 



100Z 



✓ 



6-8 



9-10 



7. 
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-6. What do you do with the print-outs of material? 
ir 2 Throw them all out. . 

2) Q tham all. 

3) 5 Save selective entries in a single file or pile 

\ 4) Q Save selective entries in separate^-iles (please 

ekplain filing system below: by subject, author, 
group f or what). 

tj i use a CRT and do not generate print-outs 

• * 

Other (Please describe) 



How many different people dd you feel you are actually exchanging 
Information or communicating with on this system, currently? 



8. Of these^ how many have you "met" (gotten to know) over the 

system? * 

9. Have you .sent transcripts or other material to persons outside the 
system, invited other persons to be Informal "observers" or 
otherwise expanded participation beyond your user group? (please 
explain)* . 




yes = 2 
no = 14 



10. 



At the present time, which of the following best describes your 

group? , 

Z More of a collection of individuals than a research community. 

G " A stt of cliques or subgroups with interests and activities 

in common, but not an integrated community ' 
3 A well integrated research community that shareslniany interests 
" and activities In common * 
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II. OVERALL REACTIONS TO THE MODE OF COMMUNICATION 

These, questions relate, to your overall reaction* to "the system, at 
this point, as a mean* of communication and work csordlnation for your 
user group* They-cpnsist of a number of rating scales on which you are ^ 
to circle one number which corresponds to where you would^plade your -own 
impressions of the system on that dimension. For example, here is the 
first scale:, 

1 1 
^J.. v Overall, the communication system is. 



Extreme- 
ly Good 

•1 



Neutral 



5 ~ 



2 - 



Extreme- 
ly Bad 

. 0 



If you think that the system is extremely good, you should circle 1. 
If you think t the system is quite good, you should ciifcle "2"; 3 would 
mean that the good aspects slightly outweigh the bad aspects. "4", the 
middle point, should be checked only when the words at the two ends of 
the scale describe the system equally well. Continuing on, "5" would 
mean that you feel that the bad aspects slightly outweigh the good as- 
pects, etc. • „ 



COL/CODE 



32 



mean=2.9 



I find using Che 

- 0 

• 

2. : 1:2 


system to 

: 3 


be 

: 4 : 


5 


1 

: 6 


1 

: 7 : 


33 


Q 

X %) • *? 


Stimula- 
ting 




Neutral 






Boring 

i 

«► 






1 4 

3. 1 :A2 


9 

: 3 


: 4 


i 
5 


1 

: 6 


0 

: 7 : 


34 


x=3.1 


' Productive 
** 










Unproduc- 
tive 




\ 


0 4 
4. : 1:2 


5 

: 3 


:' 4 : 


3 
5 


\ 

: « 


2 

: 7 : . 


. / 35 


x=2-9 


Great 
Fun 

5 6 
5. : 1:2 


t 

: 3 


0 

: 4 : 


1 
5 


•0 
: 6 


Unpleasant 

Work 

: 7 ■ 


if 

36 


x=2.3 


Time- 
Saving 










Time-. 
Wasting 


t 




4 2 
6- : 1 : % 2 


& 

': 3 


: 4 : 


3 
5 


2 

': 6 


2 

: 7 : 


37 


x=3.SJ 


* Not 
Frustra- 
ting 

i 

i 




0 




•* 


Frustrating 






! 

1 








/ 


* 




• 
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f ; 3 2.3 i 3 1 . 

38 X-3.9 7. : 1 ; 2 : 3 ; 4 : 5 ; 6 : 7 ^ : 

• < Friendly ^ 4 4 " 2 1 ^S" 0 " 1 

39 x-2.9 8/ ; 1 : 2 : 3 : 4 



5 : 6 ; 7 



Easy,, j Difficult 

Does using the system become so demanding of your time and energy 
that ic intrudes upon y6ur capacity to engage in other professional 
or personal activities? rt rt 

16 0' 2 1 0 0 0 . 

40 x=1.4 9. ; 1 : 2 s ' 3 : 4 ; 5 ; . 6 ' 7 : 

Not Very 
* demand-* demand- 

ing or log -or 

Intrusive * Intrusive 

. 3 

41 10. When you send a message over the system rather than writing, or 

telephoning/ would you say that recipients are generally 

More responsive to an electronic 'message. 

Less responsive 

No difference 




42 



- ■ 
1. What is the attitude of your spouse, children, or other persons with 
whom you live towards your use of the system? 



43 2. Which statement best describes your present, reaction to the system? 

(Please check only ,one) 



*1) 


0* 


I 


chink It is useless and should be discontinued 


2) 


0 


I 


think it has its uses for others but nop for me 


3) 


0 


I 


am skeptical but am giving it a try 


. 4) 


0 


I 


am basically indifferent "or neutral 


5) 


9 


I 


think that it has certain worthwhile uses for me 


6) 


9 


I 


think it is very useful in many respects 


7) 


1 


I 


think it is revolutionizing my work/communications 



processes. 



v III. REACTIONS TO SPECIFIC. FEATURES OF THE EIES SYSTEM 

1. Hov valuable or useful" do j^u currently find each of 
features^ or capabilities for your own comunicatipn activity? If 
lUve not actually used a feature, ' please check "cannpt say" 



COL/CODE 




Private 
Messages 


♦ 1 , ♦ 
Extreasly 
Valuable ! 


2 ' ' 3 
Fairly Slightly 
Useful . Oseful 

» 


Useless 


5 

Cannot 
Say 


_ •> 
10 


6 3 


0 


• 


Group 
Messages 


5 


t? ; 


.6 


1 


i 


Group, 
Conferences 


0 


, 2 


• 3 


12 


3 


Files 


* 0 


• 4 . 


0 


N 4 


6 


Text Editing 


0 


■3' 






5 


Oo-Liae 
Explanations 


0 


i 


4 


0 


C 



44 

45 

46 
47 
48 

49 



J 



Contents or suggestions about improving these features or desirable new 
features? * 



0 



■ I 




COL/CODE — 

2. Did soMoni demons trate % the system to you ia person or did you 
lean from the written materials? 

1-4 - 1) g live teacher 

* u written material only 



' 3. tow long did it take you to learn to use the system reasonably well 

*tedi an=1.3 hours 

4. Do you now find "the documentation" ( on a scale of 1 to 5) 

7 5 0 2 0 

8 , a) understandable 1.2 3 4 5 not understandable 

9 b) easy to read 1^- 2 t 3 hard to read 

6 2' 3 3 0 

10 c) well Organized 1 2 3 4 5 not well organized 

11 " r 5* Suggescl^fis for improvement of the Documentation.^ 



6. How would you .rate the performance of your group leader? 



3 4 10 0 

12 Excellent % 1 2 3 4 / 5 

(principal investigator) 

7. Do you find the language of the system understandable? 

4 7 5 2 1 

13 x=2.4 a) Understandable 1 2 3 4 5 Confusing 

14 x=2.8 b) Courteous* 1 2 3 4 5 Inhuman 

V 2 A 8 3 1 

8. Do you. find the use of the editing commands to be 

1 1 ' 3 5 4 2 

15 K£l»2 Easy to remember 1 2 3 4 5 Hard to remember 

16 x^A Easy to us* 1 < 2 3 4 5 Hard to use. 

.13 5 4 1 

Comments? 
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10* . Thinking back over your experiences so far with the system, 
how frequently have you felt. . (checkr one) 



Distracted by the mechanics of 
the System 


» c 

Always 


Almost^- 
Always 


3 

' Some- 
times 


4 

Almost 
Never 


5 

Never 


0 


1 


12 # 


: / 

5 


1 


Constrained in the types of 
contrlbutldn* you could make 


. 2 


4 


6 n 


0 




Overloaded vlth information 


0 




r» 
U 


i 11 

1 u 


A 


Able to express your views 


4 


5 


4 


i 

1 

> 


0 


Able to get an impression of 
personal contact *d.th other 
participants 


3 


10 


1 


3 


0 



11. How satisfactory do you think the system is for the following 
activities? 

* 

' COMPLETELY 
SATISFACTORY 



Giving or receiving 
22 x=2.0 information 



2 3 \=4. 0 Ptoblem solving \ 

2 4 x=4.4 Bargaining 

2 5 x-3-8 Generating ideas 

26 x-4.3 Persuasion 

1 1 x=JL5 Resolving disagree- 
ments 

2 8 x^ -fr Getting to know 

someone * 
Giving or receiving 

2 9 x=3 - 2 orders 

3 0 x=1 .9 Exchanging opinions 



1 • 2 
0 0 5 



0 



COMPLETELY 
UNSATISFACTORY 



0 , 0 



0 
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PLEASE PLACE A CHECK MARK OR X IN TH X APP R OPR IATE BOX TO INDICATE WHETHER 
^ KXCfi OF THE FOLLOWING FACTORS HAS BEEN VERY, IMPORTANT, SOMEWHAT IMPORTANT, OR COL/CODE 

NOT IMPORTANT AT ALL IN LIMITING YOUR USE OP THE SYSTEM, 

* 1-4 



REASON VERY 

' IMPORTANT 

'/ ' < 


SOMEWHAT 
IMPORTANT 


NOT 
IMPORTANT 


INCONVENIENT ACCESS TO A TERMINAL ^ 


• 6 


2 


10 5. 


A 

DOCUMENTATION LOOKED INADEQUATE ^ 1 

OR DIFFICULT * 1 


t 

0 


10 6 ■ 


THE SYSTEM IS TOO COMPLICATED 2 


4 . 


li 1 ■ 


TROUBLE WITH TELENET 13, 


6 




COST OF TELEPHONE OR TELENET i 


. 3 * j 14 


TROUBLE WITH TELEPHONE CONNECTION " r 


5 | 11 9 


HAD SOME BAD EXPERIENCES V*J i 

(SYSTEM CRASHED OR DID NOT SEEM \i . ^ ! ,11 
TO WORK CORRECTLY) * - 3 10 IJL 


I DO NOT LIKE TO TYPE °* 1 17 J 


I DO -NOT LIKE USING A COMPUTER 

SYSTEM LIKE THIS U, 2 


1-16 " * 


THERE IS NO ONE ON THIS SYSTEM WITH . ( ' 
WHOM I WISH TO COMMUNICATE A 

CREA* DEAL 0 6 11 lS 


I AM NOT VERY INTERESTED IN THE 

SUBJECTS BEING DISCUSSED V 


1 0 17 l ' 


OTHER PROFESSIONAL ACTIVITIES MUST 
TAKE HIGHER PRIORITY 


IV 

1:3 14 ¥ 


THE MESSAGES I HAVE RECEIVED 
DO NOT SEEM WORTH READING 


1 

i 

0 • 4 14 iy 


INADEQUATE 4 LEADERSHIP OF THE GROUP 




0 


1 ! 13 " 


OfHER (PLEASE DESCRIBE) 


*> 

o ■ 


0 


/ 3 



* NOW, PLEASE GO BACK AND CIRCLE THE SINGLE MOST IMPORTANT FACTOR > 'Whll H 
COMMENTS OR EXPLANATIONS? 




V. Conclusion . < 

1. Arc there any ideas which you are using or working with at present, which 
you .first learned of on the system? (Pleaae try to be specific about what 
you read and what Impact jt has - on your "9^* . ' 



2. Are you worlcing on any project3 or papers at the present time which have 

been advanced by your use of the system? (Again, .please try to give us some 
, specif ic details.) 



3. Are you coauthoring or collaborating closely with any members of your group 
at present time, using the system? If so, please describe who you 
are collaborating with, on what, and how' you are using the system in this 
effort. m 
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Are there any "new uses" you have Invented for the system, that are helping 
you In yP/ur work? These uses might not be related to £he specific 
purpose -of your group, but we would like to know about them. For example, 
you might use It to communicate with your family while away on business 
£rips. To coordinate face to face meetings or conferences with other 
members - - - 



Overall, what would you say_have been the main negative aspects of use 
of the system for your group this far? What things that you wish to accom- 
plish, have not occured, or what undesirable things have occured, that 
might be attributed to characteristics of communication over the system? 
Please explain as fully as possible. 



What tasks or activities can you suggest for your group on the system, to 
motivate participation? 



j ■ 

How lotjg did it take you to complete this questionnaire?^ 
Any additional comments? 
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' APPENDIX H 

EXAMPLES OF COMPUTERIZED REMINDERS AND THANK YOUS 
/ • - 

* '1. REMINDER MESSAGES. 

J 

1 ****** A GENTLE REMINDER***** 

1 have not yet received your foliow-up questionnaire. 
If it is in the mail, thankvp*^ 

If you have not r eceived- st*e ir need a new one, please 
message me. * ' , . _ 

And if it i» just lying around, won't you please take about 
twenty minutes and fill it out? 

Anxiously yours, 
' Roxanne 



PLEASE please PLEASE please PLEASE please 
Will you take a look around and see if you have 
the EIES questionnaire we sent to you awhile back? - 
If so , pleaAe take a few minutes to complete and 
return it to us so we may keep a systematic record of 
vour reactions to the .use of the system. If you do 
not have it available, message 974 for another, please. 
. Th£hk you. (And may the system always go wALl *for you) MA 
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2. ON-LINE THANK YOUS 



* 

********* ********** 

* GOOD PEOPLE AWARD * 

* * 

* * * 

* * * ♦ 
# * * * 

* * 4 



T Presented in Appreciation of your Outstanding Questionnaire-Completion 
I Efforts 
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YOU HAVE COMPLETED YOOR LAST QUESTIONNAIRE 1 WELL DONE I 
YOU can feel very proud; you are an EIES member in good standing < 
WE can relax; we have your data* Thank you* 
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APPENDIX. I A 



SELECTIONS PROM THE TRANSCRIPT OF THE "T^ELENET EXPERIENCES" 

^ClOU^S ART. KLEINER (ART.866) 4/21/80 Jl 1 ^ 
(OlIG.) 4/21/80 -6:|3 PM L:2 

KEYS: /WELCOME TO "TELENET EXPERIENCES"!/' 

> ■ . " . ' . * 

Welcome to CIO 11, Telenet Experiences. If you have problems with 
Telenet service, or wish to say something positive or negative, 
about Telenet's transmission, please enter a comment here. This 
conference will be printed out and mailed to Telenet's offices 
and to- members of < the Telenet} User's Group. As we 1 ear* more 
specifics about THAT process, we will let you know here as well. 
There will also be an effort to copy in comments about .Telenet 
\ made elsewhege in the^ system. 



I will be happy to answer any questions. *- Art, 866. 



(PS - - It may interest you to know that I was- disconnected, by 
Telenet twice while attempting to enter this comment.) 
:C1011 CC3 CHARLES WILLARD (CHARLES , 846 ) 4/21/80 9:14 PM 

KEYS :/MORE GUIDELINES ON* CONTENTS-REQUESTED/ 

f * 

I had a somewhat similar experience to Art's (cf. CC2) . I guess 
that the question that I have is whether jLt is intended .that 
thts conference shouIW be the place where routine, ^hough 
troublesome, experiences are reported. — I recall that there was 
a request in CHIMO recently for reports of freeze problems to be 
reported to EIES. Do you want a diary of troubles or only the 
big ones? * . 

^Ln picking up from . the earlier note about troubles even while 
working on this conference, U is especially unnerving, when I 
find that I am frozen online and then run through the routine of 
hanging *p Aid redialing, to be told by EIES: SORRY, THAT ID IS 
IN USE. CONNECTION TERMINATED. '* ' „ 

tciOll CC4 ART KLEINER (ART, 866) ' 4/21/80 9:29 PM L:9\ 
KEYS : /GUIDELINES/LET'S SEE HOW GUIDELINES" EMERGE AS WE KEEI\ 
REPORTING WHAT HAPPENS HERE/ 

A: 3 * • 

Charles, you won't want to report EVERY- incidence "of 
Telenet hassle here»J>ut you WILL want, after 2 ! weeks of chronic 
problems or something similar, to *ay you've, experienced two . # 
weeks of chronic .problems. Report what*you feel is worthy of 
note. We may at some point ask for brief responses to get some 
% idea of how OFTEN a particular malfunction is happening. But tor 
' right now we know they are happening often; we need 1) proof, in 
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the form of man* different people^expressing their experiences; 
and 2) we need to keep up' with any> new experiences that may > , 
happen along the Telenet trail. . 

: CIO 11 CC5 , PETER+TRUDY JOHNSON-LENZ (P+T,118) 4/22/80 6:59 
AN L:16 ~~ 

KEYS : /TELENET EXPERIENCES/LOCAL NETWORK OUTAGE/NOT OPERATING?/ . 



* 



WeNtried to log on£o EIES at about 3 AM EST this mornij 
We got the Telenet . message LOCAL NETWORK , XDUTAGE . We checkef 
other "201" computer ndmbers (20121, 2Q124, 20126) ancfall 
available and'we could 'connect . frying to coffnect to EtfES 
continued to ^ive LOCAL NETWORK OUTAGE. We. .tried calling long 
distance direct to Newark (from Portland, Oregon)*, and we were 
immediately connected to EIES. Trying again through lpcfcl ^ 
Telenet to connect, we got mor«BL6CAL NETCOM: OUTAGE and then 
stfries of 20^.25 NOT OPERATING;. Again we ca^ed direct to Newark 
and found EIES up and. humming. A^a*n ^ptried local Telenet and 
got more NOT OPERATING. Finally, about 6:23, AM EST we got . ' 
connected to EIES via local Te/efnet* 




While composing the last line, we got frozen on line agair 

H - by Telenet and then ,got LOCAL NETWORK OUTAGE when trying\to 

reconnect. Again the other n 201 n numbers worked. Finally 
reconnected. 



This is not at all unusual. 
:Cl0irCC6 "CAPTAIN AMERICA" 4/22/80 11;13AM L:18 
KEYS :/GI^EM HELL/ ^ 

IN THE .INTERESTS OF TRUTH JUSTICE AND THE AMERICAN WAY, LET V - ' / 

ME SUGGEST THE FOLLOWING: V ' 
M 1. HOW CAN YOU REALLY DETERMINE IF WHAT "YOU - "■..,» 

* • ARE EXPERIENCING IS REALLY TELENET PROBLEMS? 

2. COULD YOUR TERMINAL BE AT FAULT? 

3 . COULD; THE CUT OFF PROBLEM HAVE BEEN IN 
YOUR LOCAL CENTRAL TELEPHONE OFFICE? 

4. COULD THE PROBLEM. BE P$t THE CENTRAL. OFFICE. 
WHERE NJIT'lS LOCATED. 

5. CQULp ,THE EQUIPMENT AT NJIT HAVE CUT YOU • 
'OFF? IT CERTAINLY HAS BEEN DOWN MANY TlMES FOR EXTENDED 
PERIODS AND.. IS VERY UNRELIABLE HARDWARE. 

WE HAVE BB# CU* OFF AND FROZE ON LINE DIALING J) IRECT 
MAW TIMES. IT JUST TOO SIMPLISTIC TO BLAME TELENET FOR EVERY 
PROBLEM YOU ENCOUNTER. CERTAINLY THEY HAVE PROBLEMS , BUT' IDONT 
BELIEVE ANYWHERE NEAR WHAT THEY .TAKE THE* HEAT FOR. BUT IF IT 
MAKES YOU FEEL BETTER.'. . * .< ' 

:C1011 CC7 DANIEL H. CARTER .DANlC.,258) ' 4/22/80- 5:22 PM L:7. 
KEYS:/WHOS FAULT?/ . 
A: a v 

*• . .'• I'VE RDCSNTLY HAD EXPERIENCES. VERY MUCH LIKE- THOSE REPORTED BY 
Q - 118 JN CC 5.' IT'S MUOST IRRIDATING AND SO EARY TO BLAME 

ERJC , . • . . - <J5c 




TELENET SINCE THEY APPEAR TO BE THE COMMON ELEMENT WHEN THINGS 
FAIL OR^IT. I HOPE THAT, IN SOME WAY OR ANOTHER, WE ARE 
ABLE TO MORE REALISTICALLY DETERMINE WHO OR WHAT IS ACTUALLY AT 
FAULT, DURING THESE PERIODS OF FPUSTRATION. NOT THAT IT MAKES 
A GREAT DEAL OF DIFFERENCE, BUT IT COULD BE COMFORTJNTT TO 
KNOW THAT AT LEAST YOU!RE CUSSING THE RIGHT PARTY I 

:C1011 CC8 DANIEL H. CARTER (DAN C.,258) 4/22/80 5:27 PM L.2 
KEYS:/WHO?/ ' 

JUST AS I WAS SAYING, WHO IS RESPONSIBLE FOR THOSE CHARACTER 
'ERRORS IN MY 
CIO 11 CC7? 

:Ci011 CC9 CHARLES WILLARD (CHARLES, 846) 4/22/80 8:38 PM . 
L : 22 ' 

KEYS : /MQRE HELP REQUESTED/ 

I* am not 'certain that I gan resolve the conflict between wanting 
to be able to present TELENET with so-called proof and not 
wanting to k/now , in some detail not only the types of experiences 
but the numbers and degrees. 

,1 have learned somefchirig from Peter + Trudy, which is to try 
.other 201 accounts in TELENET t to get some idea where the problem 
'lies I call TELENET service when I have problems at the same 
times as P+T. Sometimes they can help, sometimes not. This 
morning, they thought that it would take a person coming into 
EIES *o ' work' to correct the problem, but I fotyid that it was 
back up again when I dialed in through TELENET abou^7:30 a.m. 



I have 'also found that it does not do too much good to be able 
to tell the TELENET customex service people that EIES itseir is 
' alive and well, as learned though direct dialing Newark, 
* although when I find ■ that EIES is also not answering direct 
' dialog.-- about »ne In>. twenty times — it at least says that 
thepjrpblem is not TELENET'S. 

v i am ignorant in these matters, and it seems to me it- might be 
helpful for someone to describe in lay terms — if there be such 
— the way that the connection between TELENET and ElES » made. 
That might provide some greater insights info the problems that 
we experience and the hope with which ,we might invest the 

^Ciaif cClO ART KLEINER (ART, 866) -4/22/80 9:51 PM L:29 
KEYS:./SOME TENTATIVE ANSWERS/ ' . 
A: .6 

This is« not the definitive answer to- your questions,- Captain 
2i!*£I' but I think it* will help, isolate. Other UCsand 



America, but I think it' will help* . 
implementors, who are following- this mav/ wish to add their 
comments. / t 
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1. Basically, we have to try to isolate it down' to the cause,' as 
P+T did above in'ccS. 



i. Enough people using enough different types of terminals have 
experienced the EXACT same symptoms (and later, after I get 
permission, I will copy some 6f the .msgs. the DCs have received 
into this * conference) that we can definitely exclude terminals 
as a cause of the disconnect. 

3. I think, the same applies to th|e local telephone office. One 
would expect the problem to be diff«pent in each one. Also, in 
the local telephone office or on the terminal, ! it would most 
likely either 1. not say DISCONNECTED or 2. be possible to dial 
in immediately after. (My guess^J 



I . . 

4. Us^lly itis possible to dial direct even when the Telenet 
link i*not operating. The imp lemen tors at NJIT have to my 
knowledge gone over the Telenet-EIES link software several times 
searching for' bugs. Maybe we could get a fuller report from 
someone on the NJIT staff? (There was one in* CHBiQ a few r weeks 
back but I agree with Dan C. and Charles that more is needed. 



5. Perhaps the equipment DOES need to be checked out. But users 
of OTHER networks have experienced the same symptoms. Including, 
in my own direct * experience, the Source and the I. P. Sharp 
network. A report by Robert BBezilla.in a recent issue of CHIMO 
confirmed this. . 

(I hope also to correspond with other members of the Telenet 
'User 1 * Group.) * 

Captain* A., to my knowledge you are, the only, person who has 
complained about b/»ing disconnected from EIES/during- a dial-in 
direct. Can you .provide more information on exactly what happens 
when you /are disconnected that way? Does it differ from being 
disconnected pver Telenet? Are there any other direct-dialers 
who have experienced anything similar? Or who have/ not? 



Thanks. A. 
:C1011 CC11 ART KLEINER (ART, 866) 
KEYS: /NOT OPERATING/ 
A: 10 



4/2?/80 9|:51 PM L:3 
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PS - I am dialed in directly while composing this, even tho 
Telenet reports EIES as "Not operating." 

ifclOll CC12 "SKEPTICAL" * 4/22/80 11:18 PM L:5 * 
KEYS : /UNTRUTHS/ ft 



One of the things that. bugs me most is when "Telenet lies, either 
as in 

Art's case/~SDove (not operating when it is), or even worse when I, 
phone \ 

the* and they tell me the trouble is that EIBS crashed, but I .can 

telljthem that EIES is up and gunning since I'd dialed in 
directly. 

That is NOT what I call customer service. 

:C1011 CC13 CHARLTON PRIC* (CHARLTON, 116) 4/23/80 12:12AM 
L:9 

KEYS : /MEAN ING OP "LOCAL NETWORK OUTAGE" IN PDX-SEA/ 

A * 5 ' - * . 

The pattern P+T report in cc5 is caused — I am informed 

repeatedly by Telenet customer service when I reach them in 
McLean with the (800) number — that the Portland-Seattle ports 
(they're linked, an<T in Poftland) are down. In the most recent 
instance I've encountered of this ( about 10 days ago) both the 
Portland-Seattle .ports (through which we're connected) and the 
New York ports (through which EIES gets a feed) were down at the 
same time. You also sometimes get a "domino effect" as when much 
of east coast telenet was knocked out by Hurricane David (+get 
nl000p'267t and then read the Chimo stories on these and other 

patterns). , Mtt „ w 

:C1011 CTM4 CHARLES WILLARD ( CHARLES ,846) 4/23/80 4:40AM 

L;7 * | 

/KEYS: /INFORMATION UNDERLOAD/ 

A* 12 

Illustrative of my inquiry for more information is CC12. The 
report" trout — TELENET "NOT .OPERATING" is, in fact, usually a 
correct statement with regard to the connection between EIES and 
TELENET. But it would be useful to know more about that 
connection. I gather that i-t can sometimes be corrected by the 
troops at McLean who are there round V he clock, and sometimes it 
cannot. , fc , , 

:C1011JW15 "CAPTAIN AMERICA" 4/23/80 8:37 AM L:ll 



KEYS: /PATTERNS/ ^ 

OK, FAIR ENOUGH r IT WA|S,MY INTENT TO GET EVERYONE TOjT 
LOOK FOR PATTERNS IN THEIR FAILURES AND NOT TAKE POT^ 
SHOT-S-rAT TELENET JUST FOR THE HECK OF IT. 

art in answer to your question, when cut off on dd * 
Direct dial we have to call $n and ask to be knocked off ^ine 

JUST LIKE EVERYONE ELSE. 

IF YOU CONTINUE ALONG YOUR PRESENT LINES OF 
INQUIRY IM SURE THE INFORMATION YOU ARE GATHERING WILL BE 
OF HELP TO EVERYONE IN SOLVING THE PROBLEMS. 



tClOll CC16 ALAN LEUSC* (AL,980) , 4/23/80 9:02 AM L:56 
KEYS :/TELENET/EIES/ INTERFACE/ . , 

Ltt me try and give you all a little tutorial on just how 
EIES and Telenet are hooked together. For , those of you that 
don't know, EIES is, running on a Perkin-Elmef minicomputer. It 
nWs 512,000 charactfcts of main computer memory and has a 
maxiau»*area for text storage ion our disks of 600/000,000 
characters. At- the , momenta we have used around 
200,000,000 characters of storage. The hardware was installed 
in 1975 so it is starting to ge* outdated. 

The EIES . computet has two programs in it. One is 
the EIES system itself and the other is the interface to 
Telenet, The local lines ( 201-645-5552 ) are tied directly 
into the EIES program. The Telenet lines are tied into the 
telenet program. ' There is a common area in the 

computers memory,_that both programs share, in which they 
communicate about each Telenet line. With this configuration 
it is pofciible for EIES to be up but NOT OPERATING for the 
Telenet risers. This would be caused by a number of 

problemsT^C One is that the Telene't program on our end fcas 
crashed ./ Tttiis is very rare, but does happen. The second Reason 
is that EIES is running very slow and does not 
communicate with the Telenet program fast enough to 4 allow a 
connection to be accepted on a Telenet line that~>as just a few 
moments ago disconnected. Usually several attempts* in a row 
will gain access to EIES. > This is generally the case durring 
the day. A third reason is that the protocol used to 
communicate by the Telenet system and our telenet program has 
been violated by either Telenet or us. This does happen and 
both sides are at fault in this area. Telenet usually tests its 
computer network late at night, usually around 2:00 am 
est. Once that starts we are labled either NOT REACHABLE or 
NOT OPERATING by , Telenet. Many times after they are done 
testing the net they don f t reset the . connection between us 
properly. psually it doea^clean itself up around 7:00 am 

est in the aornirt*. We can* reset the line by taking our 
Telenet interface program out of the system for several minutes, 
forcing Telenet to assume that we went dead. I've spoken 
with them several times about this, but the people manning the 
800 number don't know anything about our style of interface to 
Telenet and really- can't help. We now have a new salesman for 
Telenet and I plan on bringing up the subject with him. * 

The other common complaint ltf that" the. system just 
stops dead or freezes. l K believe* that this problem -is also 
.protocol related, but have yet been unable to replicace it when 



I am running test just for that problem. I can get the system 
to freeze on me, but it generally^fequires me ■ to have peen 
what I term' very hostile to the systtem. It requires that I 



have entered maAy carriage returns interdispersed with many 
break signals. r I realize that this is not what is freezing 
our msmtSj^/ That is why it is very important that you rejport 
to me, th5ugh this conference if you like, the, time of the day 
( est if you can ), the day, the EIES line" ycu were on and the 
Telenet number you called ( don't forget the area code ). 
With that information I can check the logs printed by Che 
telenet interface prjogram and try and get Isome idea of what is 
going on. If you don't, have time to drop a note in this 
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conference, then please send" me a* SEND with the above 
information. If" we can figure out just what is causing the 
disconnects with no time used, we might be able to .take 
advantage of that and thus drop our telenet rates. 

- As background information to these problems, Telenet is in 
the process of changing their net from PRIME computers to their 
own microcomputers. The new microcomputers, d.on't support 
the style of 4nterface we, are currently using to interface to 
•Telenet. Sometime in 1981 We will have to convert over to the 
style of int^rfact^hat^the micro's support. For those 
that care that wHiMft a XSj25 ityle interface. We are currently 
using the precursor interf ade^Jro X.25. 

:C1011 CC18 ALAN LEURCK (AL,980) , 4/23/80 11:28 AM L:3 

I have learned from TELENET that the Newark Telenet connect ion 

has a computer that has beetf" blowing up power supplies for the 

past two days. So that is the reason for the problems with 

Telenet latelyfa . - . , 

: CIO 11 CC19 RXOX/R HEROUX/M HEINES (NEIG,739) 4/23/80 1:49 

PM L:33 

KEYS : /D ITTOD ITTOD ITTO/ 

HELLO CONFERENCE PARTICIPANTS. , 

I AM VERY GLAD TO SEE THIS CONFERENCE UP AND - RUNNING, AS I 
FEEL THAT EXPLANATION IS CONSTANTLY NECESSARY AS PROBLEMS ARE 
NOT ALWAYS THE SAME. 

"i I "NOTE WITH INTEREST ALL PREVIOUS COMMENTS - IN MY 200 
HOURS PLUS OF EIES, DOE, DIALOG, AND BRS - SEARCHING I HAVE COME 
ACROSS ALL THE DIFFICULTIES MENTIONED. 1 

I AGREE WITH CAPTAIN AMERICA'S GENERAL DRIFT - THAT IS, 
THAT TELENET IS NOT ALWAYS TO BLAME. 

I ALSO HAVE ONE UP MY SLEEVE THAT OTHERS MAY OR MAY NOT 
HAVE EXPERIENCED IN THE PAST. TO WIT: AFTER A HEAVY RAINSTORM 
HERE IN THE PROVIDENCE, RI AREA OR ANYWHERE IN THE IMMEDIATE 
VICINITY (IN THIS SENSE FROM NEW JERSEY TO MAINE) I OFTEN <HAVE 
OUTAGES ON ALL SYSTEMS OVER TELENET. 

GRIPE #2: MORE P*ORTS ARE NEEDED. BETWEEN THE HOURS OF 1-4 
DAILY I AM USUALLY UNABLE TO COME ONLINE WITH ANY REGULARITY. IF 
THE BIG BENEFIT OF THE SYSTEM IS ASYNCHRONOUS COMMUNICATION, 
THEN THAT IS LOST FOR ROUGHLY HALF OF EVERY BUSINESS 
DAY . CYBEPPET ALMOST NEVER HAS THIS PROBLEM, AND WE USIjn IT 
^FREQUENTLY AS WELL. (OUCH!) / 
^. j *3. MORE PORTS ARE NEEDED AWAY FROM COSMOPOLITAN AREAS. WE 
USE TELENET /EIES AS INTEROFFICE COMMUNICATION BETWEEM PROVIDENCE 
AMD OUTLYING OFFICES IN NORTHERN. NEW HAMPSHIRE AND 
MASSACHUSETTS. IN ORDER TO PREVENT OUR EMPLOYEES FROM THROWING 
THEIR DUMB TERMINALS INTO THE ANDROSCOGGIN RIVER, IT WILL BE 
ABSOLUTELY ESSENTIAL TO HAVE MORE AND CHEAPER DATA CONNECTS IN g 
MORE WIDELY DISPERSED AREAS. 

I MUST ADMIT, I HAVE GOTTEN REASONABLE TO GOOD RESPONSE 
FROM THE TELENET SERVICE FACILITY IN VIRGINIA. ONE MINOR THING - 
ONCE, WHEN THE SERVICE PERSON HAD DETERMINED THAT THE PROBLEM 
WAS ON MY END IMPROVIDENCE, HE GAVE UP. I STILL DIDN'T HAVE 
ACCESS, AND SINCE I DIDN'T FEEL THAT IT WAS MY JOB TO CALL 
AROUND ON TELENET'S BEHALF, THE PROBLEM MAY STILL EXIST. 

A BILLION DOLLAR INDUSTRY THIS( YEAR WITH UNLIMITED GROWTH 
POTENTIAL? I HOPE SO. 

• ' ■ '• \ 
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/WILL, BACK TO THB CHIMPANZEE/TYPEWRITER x INTERFACE. 

\ MIKE 

HEINES * • 

. NEIG 

:C1011 CC25 DOUGLAS A. CAYNE (DOU.GAL, 218 ) - 4/28/80 12:28 AM • 
L:44 H 

KEYS: /GLAD TO SEE THIS CONFERENCE/MY HISTORY OP BEINp FROZEN-ON/ 

The extent of tiow frequently TELENET is freezing people on 
lately is all the more gainfully driven home by the number of 
people who have been frozen, on just while reading this 
conference. This, my first time accessing Q1011, I Was 
frozen-on in the middle of printing out ccl6 r^ha^is the first 
time I have been frozen-on while receiving outputTT^ani usually 
frozen on while the system is waiting for itoput, i.e., while I'm 
in the scratchpad or while I'm answering a- CHOICE? prompt. 

I ail frozen-on roughly 60-70% of the time I access EIES, 
which is an average /of about 3. times a day. I live on a 
borderline between area codes/ and thus can use both the Palo 
Alto, CA and San Jose,,CA TELENET otfices as a local call. Very 
o'ften, when one freezes me on and will not allow me to 
re-connect, - the other operates perfectly. Sometimes they both 
go out simultaneously. 

The usual pattej« of my experiences is that i will be 
frozen on, hangup and redial the same TELENET number, and get no 
response to c20125 for about 5 minutes.- After those 5 minutes, 
I will Often get 201 25 NOT OPERATING for about 2 minutes before 
I can again be connected. Once connected, my id is usually 
ALREADY IN USE. To combat this, I have -+STO (Set Time Oat) to 
five minutes, wh^ch reduces my ' dependency on EIES personnel to 
bump me off. But it has the adverse side-effect of forcing me 

4 to type something at least once every five minutes to avoid 

» being signed off. 

Although I have little explanation fot most of my being 
frozen on (other than what I have learned from this ' conference ) , 
when I am using EIES around 2am, either loca; Californian or 
' Eastern time, most freezing-on seems to .be due to TELENET going 
£own for maintenance of what have you'. When they shut down, 
ie*s much like being frozen on, but you can't re-dial. I'd 
think it would be possible for TELENET .to broadcast a message 
saying they will shut down in 5 minutes, rather than simply 
bumping everyone who happens to be orfwithout allowing them time 
to finish what they are doing- and sign off. 

1 I have been extremely dissatisfied with Tl&ENET lately, and 

am all the more perplexed because' I ijever had any of these 
problem* even once until just a few months ago 

I'm/glad to see this conference providing a place to air 
our complaints and to discuss ways to deal with these problems, 
j I'm grateful to Al for his cl*ar presentation of the basic 

hardware/software issues involved.' I also agree wholeheartedly 
with Richard Holbrook's suggestipn of providing users with an 
alternate ID-form that would have EIES bump the ID if it is 
currently in use; the software implementing sUch an alternate 
r D T r .code could easily be structured to prevent someone actually and 



currently ufcing. (i.e., currently typing or reading) the number 
from being bumped. 

* 

I'd be interested in hearing any ideas about why these 
problems should have started and become so severe in the past 
couple months while they were practically non-existent up until 
now ... [ 

:C1011 CC26 R COX/* HEROOX/M HEINES (NEIG,739) 4/29/80 1:34 
PM L:7 I ' 

KEYS: /RELIABILITY/ 

IT HAS OCCURRED TO ME, SOMEWHAT AS AH AFTERTHOUGHT WHILE 
DISCUSSING CC WITH NON-CC TYPES, THAT THERE' IS ONE THING THAT IS 
VERY BASIC TO THIS DISCUSSION. REGARDING NON-CC TYPES WHO COULD 
BE POTENTIAL CCERS: THEIR MAIN CRITICISM IS THAT CC HAS NOT 
SHOWN THEM THAT IT IS AS* RELIABLE AS MAKING A SIMPLE TELEPHONE 
CALL. HOW MANY PEOPfeE HAVE BEEN FROZEN IN MID-WORD WHILE MAKING 
A TRANSATLANTIC TELEPHONE CALL? COMMENTS, PLEASE. 

MIKE 

HEINES 

■ NEIG 

:C1011 CC28 MURRAY TUROFF (MURRAY, 103) 5/ 1/80 x^5:29 PM L:28 
KEYS : /RELIABILITY/ 
A: 26 

With respect to / computer services over termnals in general 
the question of reliabilty is all over the map. Computers \ do 
crash and this generation is still very subject to that soit\of 
thin* regardless of the service. There are a few systesm 
that have made the proper investment in reliablit^ and I 
think GE Time Sharing has a very good record in this regard 
from what I have heard. With sufficent funds »the problem can be 
i taken care of but the results will be reflected in user costs. 

EIES baa actually gone a. long way towrd reliabiity in the 
sense of recovering froa errors in a few seconds and almost 
total automatic operation fro long periods of time. Most of 
our prolbeM at the moment are with, the . changing nature of 
teleent aiiS the fact that they do not tell us any more about 
what they are doing therv they tell most of you. Also if any of 
their changes are giving problems to our software then we can 
hot reprogram in a few hours, it may take weeks • 

One solution ' is for us to pay another. 12,0&0 a year and 
use therre hardware interface which then' has the merit of 
putting * the whole responsi&ilty . upon them for these problems 
with out quesiton. • That t means raising our teleent 

charges. If the situatioK keeps up we will have no other 
course. ^ 

•• • . t 

EIES itself is a pilot system devolted to field trials of 
this technology and to exploring it. It is always going to have 
somewhat less reliabity then the properly deisgned commercial 
system. On. the other hand, it is about a half the costs of 
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using such a ysstem. 



In terms' of the usets we would like to see being able to 
try the system and make uie of it. I think that keeping the xost 
low has to remain a dqminen^ factor. ^ 

Yes phone calls are more reliable in terms of making the phone 
call whenever you want JLt. (except perhpas in certain 
countries and certain rurual area and NY city at certain 
times). However, the party your phoneing may not always be 
there and the cost of a phone call is considerably more than the 
cost of eies if it is between cities. 

On a cost "basis eis is more comparable to mails and possibly has 
as good reliablity. . 

/ 

There is a very dedicated group on this end which does its best 
but, one has to relaize that the degree of reliabity of. a system 
is still a strong function of the cost once you get over 
the 90% mark. I think we are over that mark at least. 
:C1011 CC29 DAVID L. JONES (DAVE J, 755) 5/ 2/80 12:34 AM L:36 
KEYS: /CAN'T GET ON TELENET SOMETIMES/ 

Just got the printout of the first 28 comments in this 
conference, and believe it or not, wasn't tossed off the systeiM? 
during all that time. There's one thing that has been happening / 
lately here in Hawaii that I don't believe^lryone has mentioned 
thus far (though I might have missed it — have' just scanned the 
printout quickly)'. r On several occasions during the past week/ I 
have dialed TELENET/ got the tone OK, but on doing the 

{CR1;(CR1 sequence absolutely nothing happens. This first 
happened on Saturday afternoon, April 26, when I was attempting V 
to demonstrate the system to a. friend. I wondered if something 
was wrong with my terminal or modem, but doubted that seriously, 
since I have an automatic dialer thru which I can listen to the 
sounds, and I could hear the pulses when I was typing. Saturday 
evening I got on EIES with no difficulty. Then when I tried ^ 
again Monday evening, the same thing happened— got TELENET tone, * 
but no other response. The -next day I < phoned John Southworth 
(HAWAII) to ask him if he'd had that happen, and he said it' had, 
several times. Tuesday and Wednesday evenings (Hawaii time) I 
tried several times during the course of each evening, and bptft 
evenings consistently got the 201 25c NOT OPERATING reply. So 
this is the first time I've been on EIES this whole week! The 
only time I have ayailable is in the afternoon ajpd evening, 
Hawaii time, which would be sometime between 11:00 p.m. and 3:00 
a.m. Eastern Daylight Time. Usually this is very good, since 
normally there are very few on-line and response time ±s 
excellent. 

Incidentally, most of the times when, I've jDeen cut off in 
mid-sentence it has been when I've been in the scratchpad 
composing a message or conference comment; only once or twice 
have I been cut off when receiving anything. And when cutoffs 
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occur/ they are quite sudden and without any * abuse - on my 
*part--i.e., no breaks'or multiple {CRl's. I usually try a 
{BREAKS just to see if I'm really cut off, after an unusually 
long pause with nothing happening. If {BREAK* doesn't print, 
then I know I'm really off. I, too, have set my Time Off to 5 
minutes, so I wouldn't have to wait 20minutes to get back on. 
This has been no, hardship; I don't have to -type anything every 5 
minutes — if I'm receiving a printout, it has the same effect. I 
can't think of any occasion when I've been on the system that I 
have been inactive for* 5 minutes at a time; 3 minutes might be 

even better, com* to ttiink of it. 

:C1011 CC31 PETER+TRUDY JOHNSON-LENZ (P+T,118) 5/ 2/80 2:33 
PM L:46 

KEYS: /MANY PROBLEMS/, 

About 4:45 AM we were trying to transmit one message from 
our micro to EIES. We logged on the system and after the normal 
log in sequence got ***POSSIBLE DATA LOSS*** which was strange 
because we weren' t transmitting yet. We disconnected and tried 
to come back again. EIES didn't recognize our number and code 
this time, so we entered it again. It worked the second time. 
We finally got into the SP and transmitted the message^, (We've 
been transmitting without transmission errors for almd^t a year 
now.') We noticed that an extra line "had crept in somewheoe, so 
we checked the message after it was sent and found it had some 
control and other garbage characters. We then copied the 
mesifcfe into the SP for editing, and when we asked the system to 
go to line 2 (with «2) , it printed out the contents'of lines 2-6 
without moving us to line 2. Then another freeze when the 
system didn't do anything. We got back to Telenet and tried to 
reconnect. Again EIES didn't recognize our number and code the • 
first time but did the secpnd. 

f We decided to try to transmit the message over again, to 

try to eliminate the transmission errors. We did, and there 
were more control and other funny characters. Again we got 
♦♦♦POSSIBLE DATA LOSS*** so we disconnected. We reset our micro 
to start over and dialed another local Telenet number and then 
- tried again. Again EIES recognized our name&code only on the 
second try, and we again got frozen on line with nothing 
happening. We finally decided to give up on fixing the message 
after half an hour of this and decided to get our waiting 
messages. We got one and part of a second (both dealing with 
someone's else's similar Telenet problems) and then the system 
froze again. This time' when we disconnected and tried to 
reconnect, we got 20125 LOCAL NETWORK OUTAGE. At this point we 
gave up. 

Throughtout this whole exper iencey-'EIES was up and there 
were people on line working. We had used the system with no 
difficulty early in the evening, so we conclude that the later 
problems were all due to Telenet. f 

jL Many of thme problems have just recurred 

during the last half hour trying to transmit this and a couple 
of messages. In fact, in adding this current note, we used 
control x to cancel a line and immediately got TELENET and the 8 
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sign. When we entered c 20125, we were right back in the SP. 
We had the saae difficulty with having to log on twice to get in 
and bad several experiences of 'POSSIBLE DATA LOSS*. Again, to 
try to prevent the problems, we reloaded our transmit program, 
hit the phone receiver a couple of times to rearrange the 
crystals, and, called Telenet anew. These problems make it 
really difficult to get much work done. Even as we entered this, 
last line, the system froze, we got back to Telenet,, and then 
got 201 25 NOT RESPONDING. After calling another Telenet 
number, we finally got reconnected. Not wanting to pus^o^ir 
luck any further, we will refrain from further comment* 
:C1011 CC33 PETER+TRUDY JOHNSON-LENZ (P+T,118) 5/ 2/80 \8 
PN L:50 

KEYS: /SWITCHED PACKETS /F EAR AND LOATHING ON TELENET/ 
A: 31 * 
Here is another example of a Telenet problem. It looks 
like some packets got switched somewhere along the line. 

i We were in the SP on line 1. We entered <a personally 
defined command r ^consult, to go through a set of conferences. 
Instead of the command executing, we got a line feed and then 
EIBS responded with COMPOSE CHOICE?. We then entered ++8 to get 
back in the SP. Upon entering, we were on line 2. Being 
curious, we entered :1 to print out line one. But the system 
froze and nothing happened. After a while we pressed BREAK and* 
got ***SOSSIBLE DATA LOSS***. Then we disconnected with and 
then reconnected. 

When we got back in the SP, line 1 was printed out. It 
.said PASSWORD (PASS) ? ~ WHICH IS NOT FROM EIESIi! Where did 
that come from?? 

Several times earlier today, we have entered commands and 
gotten weird results. For example, on as 110 long* enough to 
bump 118 because of being stuck on line, we entered +o and 
immediately got NO MESSAGES WAITING as the response. 

Just now, in trying to print out the SP witjh :- the system 
froze again. After a long pause, we pressed CR and got 
***POSSIBLE DATA LOSS***. So we disconnected and then 
reconnected. And this is the new "garbage" we f ound *in the SP * 
from out of nowhere: % 

18:MER NAME LINE 2 BELL TELCO 

19: * 

CUSTOMER* ADDRESS LINEl. 8 SOUTH 2ND AVE 
20: 

customer Address jjxmt: ~" 

21: . 
CUSTOMER « 

i V 

Then we tried to delete lines 18 on with *18- and got line 
19. We €rled to delete them again. Then on line 18 we entered 
■10 and got ***pdl*** again. So we disconnnected and then 
reconnected." When we came back in this time, after the usual 
log on, we got this random packet: 
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TI - ANKYLOSING SPONDYLITIS AND INFLAMMATORY BOWEL 
DISEASE. II. 

- ' ; • 

PREVALAENCE OF PERIPHERAL 'ARTHRIT 

% Thep we disconnected again and got 201 25 NOT RESPONDING. 
Then we got connected but had to log in twice (as mentioned ir 
cc31) since the first time didn't "take." After the usual E7 
signon messages we got 201 25 DISCONNECTED (with no time or 
packet numbers). Then we tried to reconnect and^got 201 25 NOT 
RESPONDING. Then got connected and again had to try twice. - — - 

Question: whose packets did we get and where did ours go? 

" If we weren't so addicted and persistent, we would have 
given 4p long ago. . 
:C1011 CC35 DOUGLAS/ A. CAYNE (DOUGAL,218) 5/ 3/80 5:53 PM 

L:21 / 

Of the two TELENET exchanges I use regularly,, one has been 
cutting me off all day. Rather Jthan simply freezing me\on line, 
the TELENET carrier just went dead and I was disconnected. But, 
was not frozen onto EIBS. That is, I could redial and not get 
Sorry, that ID is already in use. 

In re Dave J's cc29>about sometimes getting no response to 
the opening (CBl;4CRl, I have that problem fairly often. I've 
found, however, that typing a string of con*tol-Q's will produce 
a bit of gibberish, after which TELENET will response preperly 
to the {CRi;{CR>'. 

Does anyone have any idea why TELENET problems have 
increased so dramatically in recnent months? Is it simply ^ 
because the level of demand and use so greatly exceeds what 
TELENET expected? Are people having similar problems on other 
value-added networks? And if the problems are primarily due! to 
overload, does TELENET have any immediate plans for expanding 
their services to accomodate the greater load? 

If the networks can do no better than offering this sort of 
consistently poor— borderline unusable— service, it. may be many 
more years than we've been predicting before we become the - 
Netwfrk Nation, or 'before people find computers useful enough to^ 
have in the home... . , , „ ,« 

: CIO 11 CC46 ALAN LEURCK (AL,980) 5/ 6/80 11:00 AM L:18 
KEYS: /TELENET/NET/CHANGES/FOR/THE/BETTER???/ <v 

Well I got a little bit of a run down from one of the \ 
technical support people in Telenet. We covered two areas. The 
first was the pprtland oregan area. The problem there seems to 
have been tracked down to the dial in ports only. It seems that 
there was some problem with terminal handling software that was 
completly screwing up some of the ihportant fields as to who was 
attached to where and the type of terminal that they were using. 
The software was corrected ( seems to have been a bug ) and I 
think that all is working well now. » , 

The other problem has been in Newark. Seems that. some or 



tht ho»t computers in New Jersey were using a very dumb 
interface to Telenet that required Telenet to take all the 
characters typed into the net and pack then together into 'a 
single unit and send it out over their net. This effort * 
required a lot of computer muscle. EIES is a-Lso tied into the 
same computer that these other hosts are using. Apparently so 
much muscle was required that they wei?e unable to handle the 
entire load under heavy conditions. The very dumb interfaces « 
-have been removed from the computer we use and placed on their 
own computer. So hopefuly a large^iumber of our problems should 
do away. Please keep you commerces comming l 
-tcToil CC47 (ANONYMOUS) 5/ 6/80 1:08 PM L:0 
KEYS: /THAUK YOU FOR YOUR EFFORTS/ 
A: 46 

:C1011 CC49 EDWARD M. "HOUSMAN (GTE, 215) 5/ 8/80 10:54 AM L:4 

TELCO ALERTED ME TO THIS CONFERENCE, AND I WILL BE DROPPING IN 
FROM 

TIME TO TIME. NOW THAT THE GTE CORPORATION IS POURING MILLIONS OF 
DOLLARS INTO TELENET AND RELATED TELECOMMUNICATIONS BUSINESS AREAS 
TELENET SERVICE SHOULD CONTINUE TO IMPROVE. 
:C1011 CC51 "PANC.HO" 5/ 9/80 3:50 PM L: 3 

RE:CC 49. "SHOULD CONTINUE TO IMPROVE" WHAT IS THIS SUPPOSED TO 

^SOUNDS LIKE MIDDLE MANAGEMENT DOESN'T KNOW WHAT'S HAPPENING TO THE 
TROOP IES. 
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